Add Together AI / serverless LLM API integration example

Author: YasharF

.env.example

@@ -71,6 +71,9 @@ STEAM_KEY=D1240DEF4D41D416FD291D0075B6ED3F
 STRIPE_SKEY=sk_test_BQokikJOvBiI2HlWgH4olfQ2
 STRIPE_PKEY=pk_test_6pRNASCoBOKtIshFeQd4XMUh
 
+TOGETHERAI_API_KEY=sample-api-key
+TOGETHERAI_MODEL=meta-llama/Llama-3.3-70B-Instruct-Turbo-Free
+
 TRAKT_ID=trakt-client-id
 TRAKT_SECRET=trackt-client-secret
 

README.md

@@ -85,7 +85,13 @@ I also tried to make it as **generic** and **reusable** as possible to cover mos
 - Contact Form (powered by SMTP via Sendgrid, Mailgun, AWS SES, etc.)
 - File upload
 - **API Examples**
-  - Facebook, Foursquare, Tumblr (OAuth 1.0a example), Github, Steam, Quickbooks, Paypal, Stripe, Twilio (text messaging), Lob (USPS Mail), HERE Maps, Google Maps, Google Drive, Google Sheets, Alpha Vantage (stocks and finance info) with ChartJS, Last.fm, New York Times, Web Scraping, Trakt.tv (movies/TV)
+
+  - **AI:** OpenAI Moderation, Together AI foundational model LLMs (aka Deepseek, Llama, Mistral, etc.)
+  - **Backoffice:** Lob (USPS Mail), Paypal, Quickbooks, Stripe, Twilio (text messaging)
+  - **Data, Media & Entertainment:** Alpha Vantage (stocks and finance info) with ChartJS, Github, Foursquare, Last.fm, New York Times, Trakt.tv (movies/TV), Twitch, Tumblr (OAuth 1.0a example), Web Scraping
+  - **Maps and Location:** Google Maps, HERE Maps
+  - **Productivity:** Google Drive, Google Sheets
+
 - Flash notifications
 - reCAPTCHA and rate limit protection
 - CSRF protection
@@ -407,6 +413,16 @@ The OpenAI moderation API for checking harmful inputs is free to use as long as
 
 <hr>
 
+<img src="https://i.imgur.com/dOCkJxT.png" height="50">
+
+- Visit <a href="https://www.together.ai" target="_blank">Together AI</a>
+- Sign in or create a Together AI account.
+- Click on **Create API Key** to generate a new key. You will also be able to access your API key under your account settings in the API Keys tab.
+- Copy and paste the generated API key into your `.env` file as `TOGETHERAI_API_KEY` or set it as an environment variable.
+- Go to Together AI's <a href="https://api.together.ai/models" target="_blank"> Models</a> page and pick a model based on your usecase and budget and specify it the `TOGETHERAI_MODEL` in your `.env` file or as an environment variable (e.g. `togethercomputer/llama-3-70b-chat`).
+
+<hr>
+
 <img src="https://i.imgur.com/QMjwCk6.png" height="50">
 
 - Sign in at <a href="https://developer.x.com/" target="_blank">https://developer.x.com/</a>

app.js

@@ -232,6 +232,8 @@ app.get('/api/quickbooks', passportConfig.isAuthenticated, passportConfig.isAuth
 app.get('/api/trakt', apiController.getTrakt);
 app.get('/api/openai-moderation', apiController.getOpenAIModeration);
 app.post('/api/openai-moderation', apiController.postOpenAIModeration);
+app.get('/api/togetherai-classifier', apiController.getTogetherAIClassifier);
+app.post('/api/togetherai-classifier', apiController.postTogetherAIClassifier);
 
 /**
  * OAuth authentication routes. (Sign in)

controllers/api.js

@@ -1488,3 +1488,120 @@ exports.postOpenAIModeration = async (req, res) => {
     input: inputText,
   });
 };
+
+/**
+ * GET /api/togetherai-classifier
+ * Together AI / LLM API Example.
+ */
+exports.getTogetherAIClassifier = (req, res) => {
+  res.render('api/togetherai-classifier', {
+    title: 'Together.ai/LLM Department Classifier',
+    result: null,
+    error: null,
+    input: '',
+  });
+};
+
+/**
+ * POST /api/togetherai-classifier
+ * Together AI API Example.
+ * - Classifies customer service inquiries into departments.
+ * - Uses Together AI API with a foundational LLM model to classify the input text.
+ * - The systemPrompt is the instructions from the developer to the model for processing
+ *   the user input.
+ */
+exports.postTogetherAIClassifier = async (req, res) => {
+  const togetherAiKey = process.env.TOGETHERAI_API_KEY;
+  const togetherAiModel = process.env.TOGETHERAI_MODEL;
+  const inputText = (req.body.inputText || '').slice(0, 300);
+  let result = null;
+  let error = null;
+
+  if (!togetherAiKey) {
+    error = 'TogetherAI API key is not set in environment variables.';
+  } else if (!togetherAiModel) {
+    error = 'TogetherAI model is not set in environment variables.';
+  } else if (!inputText.trim()) {
+    error = 'Please enter a message to classify.';
+  } else {
+    try {
+      const systemPrompt = `You are a customer service classifier for an e-commerce platform. Your role is to identify the primary issue described by the customer and return the result in JSON format. Carefully analyze the customer's message and select one of the following departments as the classification result:
+
+Order Tracking and Status
+Returns and Refunds
+Payments and Billing Issues
+Account Management
+Product Inquiries
+Technical Support
+Shipping and Delivery Issues
+Promotions and Discounts
+Marketplace Seller Support
+Feedback and Complaints
+
+Provide the output in this JSON structure:
+
+{
+  "department": "<selected_department>"
+}
+Replace <selected_department> with the name of the most relevant department from the list above. If the inquiry spans multiple categories, choose the department that is most likely to address the customer's issue promptly and effectively.`;
+
+      const response = await fetch('https://api.together.xyz/v1/chat/completions', {
+        method: 'POST',
+        headers: {
+          'Content-Type': 'application/json',
+          Authorization: `Bearer ${togetherAiKey}`,
+        },
+        body: JSON.stringify({
+          model: togetherAiModel,
+          messages: [
+            { role: 'system', content: systemPrompt },
+            { role: 'user', content: inputText },
+          ],
+          temperature: 0,
+          max_tokens: 64,
+        }),
+      });
+
+      if (!response.ok) {
+        const errData = await response.json().catch(() => ({}));
+        error = errData.error && errData.error.message ? errData.error.message : `API Error: ${response.status}`;
+      } else {
+        const data = await response.json();
+        const content = data.choices && data.choices[0] && data.choices[0].message && data.choices[0].message.content;
+        let department = null;
+        if (content) {
+          try {
+            // Try to extract JSON from the response
+            const jsonStringMatch = content.match(/{.*}/s);
+            if (jsonStringMatch) {
+              const parsed = JSON.parse(jsonStringMatch[0].replace(/'/g, '"'));
+              department = parsed.department;
+            }
+          } catch (err) {
+            console.log('Failed to parse JSON from TogetherAI API response:', err);
+            // fallback: try to extract department manually
+            const match = content.match(/"department"\s*:\s*"([^"]+)"/);
+            if (match) {
+              [, department] = match;
+            }
+          }
+        }
+        result = {
+          department: department || 'Unknown',
+          raw: content,
+          systemPrompt, // Send the sysetemPrompt to the front-end for this demo, not actual production applications.
+        };
+      }
+    } catch (err) {
+      console.log('TogetherAI Classifier API Error:', err);
+      error = 'Failed to call TogetherAI API.';
+    }
+  }
+
+  res.render('api/togetherai-classifier', {
+    title: 'TogetherAI Department Classifier',
+    result,
+    error,
+    input: inputText,
+  });
+};

views/api/index.pug

@@ -133,7 +133,13 @@ block content
             | trakt.tv
     .col-md-4
       a(href='/api/openai-moderation', style='color: #000')
-        .card.mb-3(style='background-color: #f5f5f5')
+        .card.text-white.mb-3(style='background-color: rgb(36, 36, 36)')
           .card-body
             img(src='https://i.imgur.com/E0QdCiI.png', height=40, style='padding: 0px 10px 0px 0px')
             | OpenAI Input Moderation
+    .col-md-4
+      a(href='/api/togetherai-classifier', style='color: #000')
+        .card.mb-3(style='background-color: rgb(128, 181, 255)')
+          .card-body
+            img(src='https://i.imgur.com/dOCkJxT.png', height=40, style='padding: 0px 10px 0px 0px')
+            | Together AI - one-shot LLM

views/api/togetherai-classifier.pug

@@ -0,0 +1,57 @@
+extends ../layout
+
+block content
+  .pb-2.mt-2.mb-4.border-bottom
+    h2
+      i.fas.fa-network-wired.fa-sm.iconpadding(style='color: #6f42c1')
+      | Together AI - one shot LLM API call example
+
+  .btn-group.mb-4.d-flex(role='group')
+    a.btn.btn-primary.w-100(href='https://api.together.ai', target='_blank')
+      i.fas.fa-table-columns.fa-sm.iconpadding
+      | Together AI Dashboard
+    a.btn.btn-primary.w-100(href='https://docs.together.ai/docs/inference', target='_blank')
+      i.fas.fa-info.fa-sm.iconpadding
+      | Together AI Docs
+    a.btn.btn-primary.w-100(href='https://docs.together.ai/reference/chat-completions', target='_blank')
+      i.fas.fa-book.fa-sm.iconpadding
+      | API Reference
+
+  p.text-muted
+    | This example demonstrates the use of Together.AI's LLM API to build an AI Agent that classifies customer messages for department routing. TogetherAI provides serverless access to various foundational models, including Llama, Deepseek, Mistral, and more. The hackathon starter live demo leverages the Llama-3.3-70B-Instruct-Turbo-Free model. TogetherAI's REST API is fully compatible with OpenAI's API as well as locally hosted Ollama REST API, offering the flexibility to run a locally hosted LLM instead of relying on remote API calls. Furthermore, the output is requested in JSON format, enabling us to extend the code as an AI Agent to take additional actions for the user, such as querying a tracking number from a database based on the classification.
+
+  .row
+    .col-md-8.col-lg-6
+      form(method='POST', action='/api/togetherai-classifier')
+        input(type='hidden', name='_csrf', value=_csrf)
+        .mb-3
+          label(for='inputText') Customer message
+          textarea#inputText.form-control(name='inputText', maxlength='300', rows='4', required)= input
+          .form-text.text-muted Maximum 300 characters.
+        button.btn.btn-primary(type='submit') Classify
+
+      if error
+        .alert.alert-danger.mt-3= error
+
+      if result
+        .mt-4
+          h5.text-secondary.mb-3 Classification (Routing) Results
+          .d-flex.align-items-center.mb-2
+            i.fas.fa-tag.me-2(style='color: #6f42c1')
+            if result.department && result.department !== 'Unknown'
+              span.fw-bold.text-primary.fs-4 Department:
+              span.ms-2.fs-4= result.department
+            else
+              span.text-warning Could not determine department.
+          if result.raw
+            details
+              summary Show raw model output
+              pre.mt-2= result.raw
+          if result.systemPrompt
+            details
+              summary Show system prompt
+              pre.mt-2(style='white-space: pre-wrap; word-break: break-word')= result.systemPrompt
+          if result.userPrompt
+            details
+              summary Show user prompt
+              pre.mt-2= result.userPrompt