Proposal: Long running jobs

dictionary.txt

@@ -13,6 +13,7 @@ CORS
 ECR
 GCP
 GCR
+HPC
 IAM
 OpenID
 JSON
@@ -82,6 +83,7 @@ roadmap
 scaffolded
 scalable
 serverless
+transcoding
 triages
 undeploy
 uuid

src/nav.config.ts

@@ -4,6 +4,7 @@ import {
   ClockIcon,
   CloudIcon,
   CommandLineIcon,
+  CpuChipIcon,
   GlobeAltIcon,
   HomeIcon,
   LockClosedIcon,
@@ -68,6 +69,11 @@ const buildingBlockLinks = [
     href: '/http',
     icon: ServerIcon,
   },
+  {
+    title: 'AI & Batch',
+    href: '/batch',
+    icon: CpuChipIcon,
+  },
   {
     title: 'Key Value Stores',
     href: '/keyvalue',
@@ -694,6 +700,20 @@ const fullNav: FullNav = {
       links: buildingBlockLinks.filter((link) => link.href !== '/http'),
     },
   ],
+  batch: [
+    {
+      links: [
+        {
+          title: 'Overview',
+          href: '/batch',
+        },
+      ],
+    },
+    {
+      title: 'More Building Blocks',
+      links: buildingBlockLinks.filter((link) => link.href !== '/batch'),
+    },
+  ],
   keyvalue: [
     {
       links: [
@@ -1170,6 +1190,23 @@ const fullNav: FullNav = {
         },
       ],
     },
+    {
+      title: 'Batch Jobs',
+      links: [
+        {
+          title: 'job()',
+          href: '/reference/nodejs/batch/job',
+        },
+        {
+          title: 'job.handler()',
+          href: '/reference/nodejs/batch/job-handler',
+        },
+        {
+          title: 'job.send()',
+          href: '/reference/nodejs/batch/job-submit',
+        },
+      ],
+    },
     {
       title: 'HTTP',
       links: [
@@ -1636,6 +1673,23 @@ const fullNav: FullNav = {
         },
       ],
     },
+    {
+      title: 'Batch Jobs',
+      links: [
+        {
+          title: 'job()',
+          href: '/reference/python/batch/job',
+        },
+        {
+          title: 'job.handler()',
+          href: '/reference/python/batch/job-handler',
+        },
+        {
+          title: 'job.send()',
+          href: '/reference/python/batch/job-submit',
+        },
+      ],
+    },
     {
       title: 'Key Value Stores',
       links: [
@@ -1867,6 +1921,23 @@ const fullNav: FullNav = {
         },
       ],
     },
+    {
+      title: 'Batch Jobs',
+      links: [
+        {
+          title: 'job()',
+          href: '/reference/dart/batch/job',
+        },
+        {
+          title: 'job.handler()',
+          href: '/reference/dart/batch/job-handler',
+        },
+        {
+          title: 'job.send()',
+          href: '/reference/dart/batch/job-submit',
+        },
+      ],
+    },
     {
       title: 'Key Value Stores',
       links: [
@@ -2498,6 +2569,10 @@ const fullNav: FullNav = {
           title: 'NewApi()',
           href: '/reference/go/api/api',
         },
+        {
+          title: 'NewJob()',
+          href: '/reference/go/batch/job',
+        },
         {
           title: 'NewKv()',
           href: '/reference/go/keyvalue/keyvalue',
@@ -2581,6 +2656,19 @@ const fullNav: FullNav = {
         },
       ],
     },
+    {
+      title: 'Batch Jobs',
+      links: [
+        {
+          title: 'Job.Handler()',
+          href: '/reference/go/batch/job-handler',
+        },
+        {
+          title: 'Job.Send()',
+          href: '/reference/go/batch/job-submit',
+        },
+      ],
+    },
     {
       title: 'Key Value Stores',
       links: [

src/pages/batch.mdx

@@ -0,0 +1,212 @@
+export const description = 'Running AI & Batch workloads with Nitric'
+
+# Batch
+
+Nitric provides functionality that allows you to run large-scale jobs in parallel across multiple virtual machines or compute resources. Unlike Nitric Services, which respond to real-time events (APIs, Schedules, etc.), Batch is intended to efficiently handle tasks that can be processed in batches, which means they don't need to run in real time but can be executed asynchronously. Batches can include tasks that require a lot of computing power, or access to GPU resources, such as machine learning model training, image processing, video transcoding, data processing, and data analysis.
+
+<Note>
+  Batches are currently in [Preview](/reference/preview-features) and are
+  currently only available in the following languages: JavaScript, Python, Go,
+  and Dart, using the nitric/[email protected], nitric/[email protected] or later.
+</Note>
+
+Nitric Batch is designed to be used in conjunction with Nitric Services, allowing you to run long-running, computationally intensive tasks in parallel with your real-time services. This allows you to build complex applications that can handle both real-time and batch processing workloads.
+
+Batches are deployed to cloud services such as [AWS Batch](https://aws.amazon.com/batch/), [Azure Batch](https://azure.microsoft.com/en-au/products/batch), and [Google Cloud Batch](https://cloud.google.com/batch/docs). Nitric abstracts the underlying cloud provider, allowing you to run your batch jobs on any of the supported cloud providers without having to worry about the specifics of each provider.
+
+## Enabling Batches
+
+Batches are currently in [Preview](/reference/preview-features). To enable this feature in your project add the following to your `nitric.yaml` file
+
+```yaml
+preview:
+  - batch-services
+```
+
+## Definitions
+
+### Batch
+
+A Batch is similar to a Nitric Service, but it's intended for work with a definitive start and a finish. Where a service is designed to be reactive, a batch is designed to be proactive and run a series of jobs in parallel.
+
+### Job Definitions
+
+A Job Definition describes a type of work to be done by a Nitric `Batch`
+
+### Job
+
+A Job is an instance of a Job Definition that is running within a `Batch`, Jobs can be started from other Nitric Services or Batches.
+
+## Limitations of Batches
+
+Jobs are designed to be long running HPC workloads and can take some time to spin up. They are not designed with reactivity in mind and are not suitable for responding to events from cloud resources.
+
+Jobs are unable to run the following:
+
+- Topic Subscriptions
+- Bucket Notifications
+- API & HTTP resources
+- Websocket message handlers
+
+Jobs can be used to read and write to/from all nitric resources, for example they can publish new messages to a Topic, read and write to a Bucket, or read and write to a Database. They just can't respond to real-time events from these resources.
+
+## Defining Batches
+
+Batches are defined similarly to services in a project's `nitric.yaml` file. For example:
+
+```yaml
+batch-services:
+  - match: ./batches/*.ts
+    start: yarn dev:services $SERVICE_PATH
+```
+
+<Note>Batches can contain any number of Job Definitions.</Note>
+
+## Defining a Job
+
+Within a Batch we create Job Definitions, by creating a new Job with a unique name and defining a handler function that will be executed when the job is submitted.
+
+<CodeGroup>
+
+```typescript
+import { job, JobContext } from '@nitric/sdk'
+
+const analyze = job('analyze')
+
+// Use `handler` to register the callback function that will run when a job is submitted
+analyze.handler(
+  async (ctx: JobContext) => {
+    // Do some work
+  },
+  { cpus: 1, memory: 1024, gpus: 0 }
+)
+```
+
+```python
+from nitric.resources import job
+from nitric.application import Nitric
+from nitric.context import JobContext
+
+analyze = job("analyze")
+
+# Create the callback function that will run when a job is submitted
+@analyze(cpus=1, memory=1024, gpus=0)
+async def generate_image(ctx: None):
+    # Do some work
+
+
+Nitric.run()
+```
+
+```go
+import (
+  "github.com/nitrictech/go-sdk/nitric"
+  "github.com/nitrictech/go-sdk/nitric/batch"
+)
+
+func main() {
+  analyze := nitric.NewJob("analyze")
+
+  // Use `Handler` to register the callback function that will run when a job is submitted
+  analyze.Handler(func(ctx *batch.Ctx) {
+    // Do some work
+  }, batch.WithCpus(1), batch.WithMemory(1024), batch.WithGpus(0))
+
+  nitric.Run()
+}
+```
+
+```dart
+import 'package:nitric_sdk/nitric.dart';
+
+void main() {
+  final job = Nitric.job("analyze");
+
+  job.handler((ctx) async {
+    print("New job submitted for ${ctx.req.jobName}: ${ctx.req.message}");
+
+    return ctx;
+  }, opts: JobResourceRequirements(cpus: 1, memory: 1024, gpus: 0));
+}
+```
+
+</CodeGroup>
+
+## Submitting Jobs for Execution
+
+Jobs may be submitted from Nitric `services` or other `batches` using the `submit` method on the job reference. When submitting a job you can provide a payload that will be passed to the job handler function.
+
+<CodeGroup>
+
+```javascript
+import * as nitric from '@nitric/sdk'
+
+const api = nitric.api('public')
+const analyze = nitric.job('analyze').allow('submit')
+
+api.post('/submit-job', async (ctx) => {
+  await analyze.submit({
+    someKey: 'someValue',
+  })
+})
+```
+
+```python
+from nitric.resources import api, job
+from nitric.application import Nitric
+
+analyze = job("analyze").allow("submit")
+public_api = api("public")
+
+@public_api.post("/submit-job")
+async def submit_job(ctx):
+    await analyze.submit(
+      {
+        "someKey": "someValue"
+      }
+    )
+
+Nitric.run()
+```
+
+```go
+import (
+	"context"
+
+	"github.com/nitrictech/go-sdk/nitric"
+	"github.com/nitrictech/go-sdk/nitric/apis"
+	"github.com/nitrictech/go-sdk/nitric/batch"
+)
+
+func main() {
+	api := nitric.NewApi("public")
+	analyze := nitric.NewJob("analyze").Allow(batch.JobSubmit)
+
+	api.Post("/submit-job", func(ctx *apis.Ctx) {
+		analyze.Submit(context.Background(), map[string]interface{}{
+			"someKey": "someValue",
+		})
+	})
+
+	nitric.Run()
+}
+```
+
+```dart
+import 'package:nitric_sdk/nitric.dart';
+
+void main() {
+  final api = Nitric.api("public");
+  final analyze = Nitric.job("analyze").allow([JobPermission.submit]);
+
+  api.get("/submit-job", (ctx) async {
+    analyze.submit({
+      "someKey": "someValue"
+    });
+
+    return ctx;
+  });
+}
+```
+
+</CodeGroup>