Merge remote-tracking branch 'upstream/main' into fix/stream-restore-on-thread-navigation

Author: kubet

.cursor/rules/suna-project.mdc

@@ -70,7 +70,6 @@ project/
 - **LLM Integration**: LiteLLM for multi-provider support, structured prompts
 - **Tool System**: Dual schema decorators (OpenAPI + XML), consistent ToolResult
 - **Real-time**: Supabase subscriptions for live updates
-- **Background Jobs**: Dramatiq for async processing, QStash for scheduling
 
 ## Key Technologies
 

CONTRIBUTING.md

@@ -41,7 +41,6 @@ Before contributing, ensure you have access to:
 - Daytona account (for agent execution)
 - Tavily API key (for search)
 - Firecrawl API key (for web scraping)
-- QStash account (for background jobs)
 
 **Optional:**
 

README.md

@@ -99,7 +99,6 @@ The setup process includes:
 - Setting up Daytona for secure agent execution
 - Integrating with LLM providers (Anthropic, OpenAI, OpenRouter, etc.)
 - Configuring web search and scraping capabilities (Tavily, Firecrawl)
-- Setting up QStash for background job processing and workflows
 - Configuring webhook handling for automated tasks
 - Optional integrations (RapidAPI for data providers)
 
@@ -147,14 +146,13 @@ We welcome contributions from the community! Please see our [Contributing Guide]
 ### Technologies
 
 - [Daytona](https://daytona.io/) - Secure agent execution environment
-- [Supabase](https://supabase.com/) - Database and authentication
+- [Supabase](https://supabase.com/) - Database, Cron, and Authentication
 - [Playwright](https://playwright.dev/) - Browser automation
 - [OpenAI](https://openai.com/) - LLM provider
 - [Anthropic](https://www.anthropic.com/) - LLM provider
 - [Morph](https://morphllm.com/) - For AI-powered code editing
 - [Tavily](https://tavily.com/) - Search capabilities
 - [Firecrawl](https://firecrawl.dev/) - Web scraping capabilities
-- [QStash](https://upstash.com/qstash) - Background job processing and workflows
 - [RapidAPI](https://rapidapi.com/) - API services
 - Custom MCP servers - Extend functionality with custom tools
 

backend/.env.example

@@ -54,11 +54,6 @@ SMITHERY_API_KEY=
 
 MCP_CREDENTIAL_ENCRYPTION_KEY=
 
-QSTASH_URL="https://qstash.upstash.io"
-QSTASH_TOKEN=""
-QSTASH_CURRENT_SIGNING_KEY=""
-QSTASH_NEXT_SIGNING_KEY=""
-
 WEBHOOK_BASE_URL=""
 
 # Optional

backend/README.md

@@ -96,11 +96,6 @@ DAYTONA_API_KEY=your-daytona-key
 DAYTONA_SERVER_URL=https://app.daytona.io/api
 DAYTONA_TARGET=us
 
-# Background Job Processing (Required)
-QSTASH_URL=https://qstash.upstash.io
-QSTASH_TOKEN=your-qstash-token
-QSTASH_CURRENT_SIGNING_KEY=your-current-signing-key
-QSTASH_NEXT_SIGNING_KEY=your-next-signing-key
 WEBHOOK_BASE_URL=https://yourdomain.com
 
 # MCP Configuration

backend/agent/prompt.py

@@ -533,15 +533,13 @@
 
 **CRITICAL EXECUTION ORDER RULES:**
 1. **SEQUENTIAL EXECUTION ONLY:** You MUST execute tasks in the exact order they appear in the Task List
-2. **ONE TASK AT A TIME:** Never execute multiple tasks simultaneously or in bulk
+2. **ONE TASK AT A TIME:** Never execute multiple tasks simultaneously or in bulk, but you can update multiple tasks in a single call
 3. **COMPLETE BEFORE MOVING:** Finish the current task completely before starting the next one
 4. **NO SKIPPING:** Do not skip tasks or jump ahead - follow the list strictly in order
 5. **NO BULK OPERATIONS:** Never do multiple web searches, file operations, or tool calls at once
 6. **ASK WHEN UNCLEAR:** If you encounter ambiguous results or unclear information during task execution, stop and ask for clarification before proceeding
 7. **DON'T ASSUME:** When tool results are unclear or don't match expectations, ask the user for guidance rather than making assumptions
-8. **MANDATORY TASK COMPLETION:** After completing each task, IMMEDIATELY update it to "completed" status before proceeding to the next task
-9. **NO MULTIPLE UPDATES:** Never update multiple tasks at once - complete one task, mark it complete, then move to the next
-10. **VERIFICATION REQUIRED:** Only mark a task as complete when you have concrete evidence of completion
+8. **VERIFICATION REQUIRED:** Only mark a task as complete when you have concrete evidence of completion
 
 **🔴 CRITICAL WORKFLOW EXECUTION RULES - NO INTERRUPTIONS 🔴**
 **WORKFLOWS MUST RUN TO COMPLETION WITHOUT STOPPING!**
@@ -596,12 +594,11 @@
 **MANDATORY EXECUTION CYCLE:**
 1. **IDENTIFY NEXT TASK:** Use view_tasks to see which task is next in sequence
 2. **EXECUTE SINGLE TASK:** Work on exactly one task until it's fully complete
-3. **UPDATE TO COMPLETED:** Immediately mark the completed task as "completed" using update_tasks
-4. **MOVE TO NEXT:** Only after marking the current task complete, move to the next task
-5. **REPEAT:** Continue this cycle until all tasks are complete
-6. **SIGNAL COMPLETION:** Use 'complete' or 'ask' when all tasks are finished
-
-**CRITICAL: NEVER execute multiple tasks simultaneously or update multiple tasks at once. Always complete one task fully, mark it complete, then move to the next.**
+3. **THINK ABOUT BATCHING:** Before updating, consider if you have completed multiple tasks that can be batched into a single update call
+4. **UPDATE TO COMPLETED:** Update the status of completed task(s) to 'completed'. EFFICIENT APPROACH: Batch multiple completed tasks into one update call rather than making multiple consecutive calls
+5. **MOVE TO NEXT:** Only after marking the current task complete, move to the next task
+6. **REPEAT:** Continue this cycle until all tasks are complete
+7. **SIGNAL COMPLETION:** Use 'complete' or 'ask' when all tasks are finished
 
 **HANDLING AMBIGUOUS RESULTS DURING TASK EXECUTION:**
 1. **WORKFLOW CONTEXT MATTERS:** 
@@ -672,7 +669,7 @@
 4. **EXECUTION:** Wait for tool execution and observe results
 5. **TASK COMPLETION:** Verify the current task is fully completed before moving to the next
 6. **NARRATIVE UPDATE:** Provide **Markdown-formatted** narrative updates explaining what was accomplished and what's next
-7. **PROGRESS TRACKING:** Mark current task complete, update Task List with any new tasks needed
+7. **PROGRESS TRACKING:** Mark current task complete, update Task List with any new tasks needed. EFFICIENT APPROACH: Consider batching multiple completed tasks into a single update call
 8. **NEXT TASK:** Move to the next task in sequence - NEVER skip ahead or do multiple tasks at once
 9. **METHODICAL ITERATION:** Repeat this cycle for each task in order until all tasks are complete
 10. **COMPLETION:** IMMEDIATELY use 'complete' or 'ask' when ALL tasks are finished
@@ -717,13 +714,6 @@
 - Technical documentation or guides
 - Any content that would be better as an editable artifact
 
-**CRITICAL FILE CREATION RULES:**
-- **ONE FILE PER REQUEST:** For a single user request, create ONE file and edit it throughout the entire process
-- **EDIT LIKE AN ARTIFACT:** Treat the file as a living document that you continuously update and improve
-- **APPEND AND UPDATE:** Add new sections, update existing content, and refine the file as you work
-- **NO MULTIPLE FILES:** Never create separate files for different parts of the same request
-- **COMPREHENSIVE DOCUMENT:** Build one comprehensive file that contains all related content
-
 **CRITICAL FILE CREATION RULES:**
 - **ONE FILE PER REQUEST:** For a single user request, create ONE file and edit it throughout the entire process
 - **EDIT LIKE AN ARTIFACT:** Treat the file as a living document that you continuously update and improve

backend/agent/tools/task_list_tool.py

@@ -345,7 +345,7 @@ async def create_tasks(self, sections: Optional[List[Dict[str, Any]]] = None,
         "type": "function",
         "function": {
             "name": "update_tasks",
-            "description": "Update one or more tasks. Can update content, status, or move tasks between sections. IMPORTANT: Follow the one-task-at-a-time execution principle. After completing each individual task, immediately update it to 'completed' status before proceeding to the next task. This ensures proper progress tracking and prevents bulk operations that violate the sequential execution workflow.",
+                "description": "Update one or more tasks. EFFICIENT BATCHING: Before calling this tool, think about what tasks you have completed and batch them into a single update call. This is more efficient than making multiple consecutive update calls. Always execute tasks in the exact sequence they appear, but batch your updates when possible. Update task status to 'completed' after finishing each task, and consider batching multiple completed tasks into one call rather than updating them individually.",
             "parameters": {
                 "type": "object",
                 "properties": {
@@ -354,7 +354,7 @@ async def create_tasks(self, sections: Optional[List[Dict[str, Any]]] = None,
                             {"type": "string"},
                             {"type": "array", "items": {"type": "string"}, "minItems": 1}
                         ],
-                        "description": "Task ID (string) or array of task IDs to update. For optimal workflow, prefer updating single tasks to 'completed' status immediately after completion rather than bulk updates."
+                        "description": "Task ID (string) or array of task IDs to update. EFFICIENT APPROACH: Batch multiple completed tasks into a single call rather than making multiple consecutive update calls. Always maintain sequential execution order."
                     },
                     "content": {
                         "type": "string",
@@ -363,7 +363,7 @@ async def create_tasks(self, sections: Optional[List[Dict[str, Any]]] = None,
                     "status": {
                         "type": "string",
                         "enum": ["pending", "completed", "cancelled"],
-                        "description": "New status for the task(s) (optional). Use 'completed' immediately after finishing each individual task to maintain proper execution flow."
+                        "description": "New status for the task(s) (optional). Set to 'completed' for finished tasks. Batch multiple completed tasks when possible."
                     },
                     "section_id": {
                         "type": "string",
@@ -376,15 +376,15 @@ async def create_tasks(self, sections: Optional[List[Dict[str, Any]]] = None,
     })
     @usage_example(
         '''
-        # Update single task:
+        # Update single task (when only one task is completed):
         <function_calls>
         <invoke name="update_tasks">
         <parameter name="task_ids">task-uuid-here</parameter>
         <parameter name="status">completed</parameter>
         </invoke>
         </function_calls>
         
-        # Update multiple tasks:
+        # Update multiple tasks (EFFICIENT: batch multiple completed tasks):
         <function_calls>
         <invoke name="update_tasks">
         <parameter name="task_ids">["task-id-1", "task-id-2", "task-id-3"]</parameter>

backend/pyproject.toml

@@ -58,7 +58,6 @@ dependencies = [
   "cryptography>=41.0.0",
   "apscheduler>=3.10.0",
   "croniter>=1.4.0",
-  "qstash>=2.0.0",
   "structlog==25.4.0",
   "PyPDF2==3.0.1",
   "python-docx==1.1.0",

backend/supabase/migrations/20250808120000_supabase_cron.sql

@@ -0,0 +1,96 @@
+-- Enable Supabase Cron and HTTP extensions and provide helper RPCs
+-- This migration replaces QStash-based scheduling with Supabase Cron
+
+BEGIN;
+
+-- Enable required extensions if not already enabled
+CREATE EXTENSION IF NOT EXISTS pg_cron;
+CREATE EXTENSION IF NOT EXISTS pg_net;
+
+-- Helper function to schedule an HTTP POST via Supabase Cron
+-- Overwrites existing job with the same name
+CREATE OR REPLACE FUNCTION public.schedule_trigger_http(
+    job_name text,
+    schedule text,
+    url text,
+    headers jsonb DEFAULT '{}'::jsonb,
+    body jsonb DEFAULT '{}'::jsonb,
+    timeout_ms integer DEFAULT 8000
+) RETURNS bigint
+LANGUAGE plpgsql
+SECURITY DEFINER
+AS $$
+DECLARE
+    job_id bigint;
+    sql_text text;
+    headers_fixed jsonb;
+    body_fixed jsonb;
+BEGIN
+    -- Unschedule any existing jobs with the same name
+    PERFORM cron.unschedule(j.jobid)
+    FROM cron.job j
+    WHERE j.jobname = job_name;
+
+    -- Normalize headers/body in case callers pass JSON strings instead of objects
+    headers_fixed := COALESCE(
+        CASE
+            WHEN headers IS NULL THEN '{}'::jsonb
+            WHEN jsonb_typeof(headers) = 'object' THEN headers
+            WHEN jsonb_typeof(headers) = 'string' THEN (
+                -- Remove surrounding quotes then unescape to get raw JSON text, finally cast to jsonb
+                replace(replace(trim(both '"' from headers::text), '\\"', '"'), '\\\\', '\\')
+            )::jsonb
+            ELSE '{}'::jsonb
+        END,
+        '{}'::jsonb
+    );
+
+    body_fixed := COALESCE(
+        CASE
+            WHEN body IS NULL THEN '{}'::jsonb
+            WHEN jsonb_typeof(body) = 'object' THEN body
+            WHEN jsonb_typeof(body) = 'string' THEN (
+                replace(replace(trim(both '"' from body::text), '\\"', '"'), '\\\\', '\\')
+            )::jsonb
+            ELSE body
+        END,
+        '{}'::jsonb
+    );
+
+    -- Build the SQL snippet to be executed by pg_cron
+    sql_text := format(
+        $sql$select net.http_post(
+            url := %L,
+            headers := %L::jsonb,
+            body := %L::jsonb,
+            timeout_milliseconds := %s
+        );$sql$,
+        url,
+        headers_fixed::text,
+        body_fixed::text,
+        timeout_ms
+    );
+
+    job_id := cron.schedule(job_name, schedule, sql_text);
+    RETURN job_id;
+END;
+$$;
+
+-- Helper to unschedule by job name
+CREATE OR REPLACE FUNCTION public.unschedule_job_by_name(job_name text)
+RETURNS void
+LANGUAGE plpgsql
+SECURITY DEFINER
+AS $$
+BEGIN
+    PERFORM cron.unschedule(j.jobid)
+    FROM cron.job j
+    WHERE j.jobname = job_name;
+END;
+$$;
+
+-- Grant execute to service role (backend uses service role key)
+GRANT EXECUTE ON FUNCTION public.schedule_trigger_http(text, text, text, jsonb, jsonb, integer) TO service_role;
+GRANT EXECUTE ON FUNCTION public.unschedule_job_by_name(text) TO service_role;
+
+COMMIT;

backend/triggers/api.py

@@ -6,6 +6,7 @@
 import uuid
 from datetime import datetime, timezone
 import json
+import hmac
 
 from services.supabase import DBConnection
 from utils.auth_utils import get_current_user_id_from_jwt
@@ -452,6 +453,18 @@ async def trigger_webhook(
         raise HTTPException(status_code=403, detail="Agent triggers are not enabled")
 
     try:
+        # Simple header-based auth using a shared secret
+        # Configure the secret via environment variable: TRIGGER_WEBHOOK_SECRET
+        secret = os.getenv("TRIGGER_WEBHOOK_SECRET")
+        if not secret:
+            logger.error("TRIGGER_WEBHOOK_SECRET is not configured")
+            raise HTTPException(status_code=500, detail="Webhook secret not configured")
+
+        incoming_secret = request.headers.get("x-trigger-secret", "")
+        if not hmac.compare_digest(incoming_secret, secret):
+            logger.warning(f"Invalid webhook secret for trigger {trigger_id}")
+            raise HTTPException(status_code=401, detail="Unauthorized")
+
         # Get raw data from request
         raw_data = {}
         try: