Docs - troubleshooting

docs-v2/pages/troubleshooting/index.mdx → docs-v2/pages/troubleshooting.mdx

@@ -4,10 +4,6 @@ import Callout from '@/components/Callout'
 
 This doc describes some common solutions for fixing issues with [pipedream.com](https://pipedream.com) or with a specific workflow.
 
-## How do I contact Pipedream Support?
-
-Start by filling out the request form at [https://pipedream.com/support](https://pipedream.com/support), providing detailed information about your issue.
-
 ## A feature isn't working on pipedream.com
 
 If you're seeing an issue with [pipedream.com](https://pipedream.com) (for example, the site won't load, or you think you've found a bug), try each of the following steps, checking to see if they fix the problem:
@@ -20,7 +16,33 @@ If you're seeing an issue with [pipedream.com](https://pipedream.com) (for examp
 
 If you're still seeing the issue after trying these steps, please [report a bug](https://github.com/PipedreamHQ/pipedream/issues/new?assignees=&labels=bug&template=bug_report.md&title=%5BBUG%5D+).
 
-## My workflow isn't working
+## How do I contact Pipedream Support?
+
+Start by filling out the request form at [https://pipedream.com/support](https://pipedream.com/support), providing detailed information about your issue.
+
+### How do I share my workflow with Support?
+
+First, navigate to your **Project Settings** and share your project with Pipedream Support.
+
+When filling out the request form at [https://pipedream.com/support](https://pipedream.com/support), please provide detailed information along with the URL from your browser's address bar, which should look something like:
+
+```
+https://pipedream.com/@yourworkspace/projects/proj_abc123/test-workflow-p_abc123/inspect
+```
+
+## Workflows
+
+### Where do I find my workflow's ID?
+
+Open [https://pipedream.com](https://pipedream.com) and visit your workflow. Copy the URL that appears in your browser's address bar. For example:
+
+```
+https://pipedream.com/@dylburger/p_abc123/build
+```
+
+Your workflow's ID is the value that starts with `p_`. In this example: `p_abc123`.
+
+### My workflow isn't working
 
 If you're encountering a specific issue in a workflow, try the following steps, checking to see if they fix the problem:
 
@@ -30,15 +52,21 @@ If you're encountering a specific issue in a workflow, try the following steps,
 
 If you're still seeing the issue after trying these steps, please reach out in [the community](https://pipedream.com/support).
 
-## I need help with my GitHub Synced Project
+### How do I invoke another workflow?
 
-[See FAQ here](/workflows/git/#faq).
+We provide a [Trigger Workflow](https://pipedream.com/apps/helper-functions/actions/trigger-workflow) action in the [Helper Functions](https://pipedream.com/apps/helper-functions) app. [See more here](/workflows/building-workflows/code/nodejs/#invoke-another-workflow).
+
+Another option is to make an HTTP request to a Pipedream HTTP webhook trigger.
 
-## Why is my trigger not emitting events?
+## Triggers
 
-Most Pipedream sources fall into one of two categories: webhook-based or timer-based.
+### Why is my trigger not emitting events?
 
-### Webhook-based instant sources
+First, look at your [trigger logs](/troubleshooting/#where-can-i-find-the-trigger-logs) and check for any errors there. Verify that it has been running, whether on a new webhook event or at the configured polling interval.
+
+If your trigger operates on a large amount of data at once, it may fail to log the execution and you won't see any new events and logs. Try polling more frequently or limiting the number of records being fetched in the API request or Database query.
+
+#### Webhook-based instant sources
 
 These sources will get triggered immediately. But because events come in in real-time, most will **not** automatically fetch historical events upon creation. To surface test events in your workflow while building, you'll need to generate an eligible event in the selected app.
 
@@ -48,41 +76,33 @@ For example, if you've configured the "[Message Updates (Instant)](https://piped
 
 Sources for apps like [Telegram](https://pipedream.com/apps/telegram-bot-api/triggers/message-updates) and [Google Sheets](https://pipedream.com/apps/google-sheets/triggers/new-row-added) use webhooks and get triggered immediately.
 
-### Timer-based polling sources
+#### Timer-based polling sources
 
 These sources will fetch new events on a regular interval, based on a schedule you specify in the trigger configuration.
 
 ![Configure polling timer](https://res.cloudinary.com/pipedreamin/image/upload/v1653434591/docs/polling-triggers-timer_ooz26c.png)
 
 In most cases, Pipedream will automatically fetch recent historical events to help enable easier workflow development. Sources for apps like [Twitter](https://pipedream.com/apps/twitter/triggers/search-mentions) and [Spotify](https://pipedream.com/apps/spotify/triggers/new-playlist) require we poll their endpoints in order to fetch new events.
 
-## Where do I find my workflow's ID?
+### Where do I find my event source's ID?
 
-Open [https://pipedream.com](https://pipedream.com) and visit your workflow. Copy the URL that appears in your browser's address bar. For example:
+Open [https://pipedream.com/sources](https://pipedream.com/sources) and click on your event source. Copy the URL that appears in your browser's address bar. For example:
 
 ```
-https://pipedream.com/@dylburger/p_abc123/edit
+https://pipedream.com/sources/dc_abc123
 ```
 
-Your workflow's ID is the value that starts with `p_`. In this example: `p_abc123`.
-
-## How do I invoke another workflow?
-
-We provide a [Trigger Workflow](https://pipedream.com/apps/helper-functions/actions/trigger-workflow) action in the [Helper Functions](https://pipedream.com/apps/helper-functions) app. [See more here](/workflows/building-workflows/code/nodejs/#invoke-another-workflow).
+Your source's ID is the value that starts with `dc_`. In this example: `dc_abc123`.
 
-Another option is to make an HTTP request to a Pipedream HTTP webhook trigger.
+### Where can I find the trigger logs?
 
-## Where do I find my event source's ID?
-
-Open [https://pipedream.com/sources](https://pipedream.com/sources) and click on your event source. Copy the URL that appears in your browser's address bar. For example:
+Find your [source](/troubleshooting/#where-do-i-find-my-event-sources-id), then click on the logs or visit this URL:
 
 ```
-https://pipedream.com/sources/dc_abc123
+https://pipedream.com/sources/dc_abc123/logs
 ```
 
-Your source's ID is the value that starts with `dc_`. In this example: `dc_abc123`.
-
-## Why is my trigger paused?
+### Why is my trigger paused?
 
 Pipedream automatically disables sources with a 100% error rate in the past 5 days for accounts on the Free plan.
 
@@ -92,22 +112,37 @@ To troubleshoot, you can look at the errors in the [source](/workflows/building-
 
 Pipedream displays warnings below steps in certain conditions. These warnings do not stop the execution of your workflow, but can signal an issue you should be aware of.
 
-## Limit Exceeded Errors
+### Code was still running when the step ended
 
-Pipedream sets [limits](/workflows/limits/) on runtime, memory, and other execution-related properties. If you exceed these limits, you'll receive one of the errors below. [See the limits doc](/workflows/limits/) for details on specific limits.
+This error occurs when Promises or asynchronous code is not properly finished before the next step begins execution.
 
+See the [Asynchronous section of the Node.js documentation](/workflows/building-workflows/code/nodejs/async/#the-problem) for more details.
 
-## Quota Exceeded
+### Undeployed changes — You have made changes to this workflow. Deploy the latest version from the editor
+
+On workflows that are not [synced with GitHub](/workflows/git/), you may notice the following warning at the top of your workflow:
+
+> **Undeployed changes** — You have made changes to this workflow. Deploy the latest version from the editor
+
+This means that you've made some changes to your workflow that you haven't yet deployed. To see a diff of what's changed, we recommend [enabling GitHub sync](/workflows/git/), where you'll get a full commit history of changes made to your workflows, synced to your own GitHub repo.
+
+## Errors
+
+### Limit Exceeded Errors
+
+Pipedream sets [limits](/workflows/limits/) on runtime, memory, and other execution-related properties. If you exceed these limits, you'll receive one of the errors below. [See the limits doc](/workflows/limits/) for details on specific limits.
+
+### Quota Exceeded
 
 On the Free tier, Pipedream imposes a limit on the [daily credits](/workflows/limits/#daily-credits-limit) across all workflows and sources. If you hit this limit, you'll see a **Quota Exceeded** error.
 
 Paid plans have no credit limit. [Upgrade here](https://pipedream.com/pricing).
 
-## Runtime Quota Exceeded
+### Runtime Quota Exceeded
 
 You **do not** use credits testing workflows, but workspaces on the **Free** plan are limited to {process.env.DAILY_TESTING_LIMIT} of test runtime per day. If you exceed this limit when testing in the builder, you'll see a **Runtime Quota Exceeded** error.
 
-## Timeout
+### Timeout
 
 Event sources and workflows have a [default time limit on a given execution](/workflows/limits/#time-per-execution). If your code exceeds that limit, you may encounter a **Timeout** error.
 
@@ -116,7 +151,7 @@ To address timeouts, you'll either need to:
 1. Figure out why your code is running for longer than expected. It's important to note that **timeouts are not an issue with Pipedream — they are specific to your workflow**. Often, you're making a request to a third party API that doesn't respond in the time you expect, or you're processing a large amount of data in your workflow, and it doesn't complete before you hit the execution limit.
 2. If it's expected that your code is taking a long time to run, you can raise the execution limit of a workflow in your [workflow's settings](/workflows/building-workflows/settings/#execution-timeout-limit). If you need to change the execution limit for an event source, please [reach out to our team](https://pipedream.com/support/).
 
-## Out of Memory
+### Out of Memory
 
 Pipedream [limits the default memory](/workflows/limits/#memory) available to workflows and event sources. If you exceed this memory, you'll see an **Out of Memory** error. **You can raise the memory of your workflow [in your workflow's Settings](/workflows/building-workflows/settings/#memory)**.
 
@@ -125,13 +160,13 @@ This can happen for two main reasons:
 1. When you load a large file or object into the workflow's memory (e.g. when you save the content in a variable). Where possible, consider streaming the file to / from disk, instead of storing it in memory, using a [technique like this](/workflows/building-workflows/code/nodejs/http-requests/#download-a-file-to-the-tmp-directory).
 2. When you have many steps in your Pipedream workflow. When your workflow runs, Pipedream runs a separate process for each step in your workflow. That incurs some memory overhead. Typically this happens when you have more than 8-10 steps. When you see an OOM error on a workflow with many steps, try increasing the memory.
 
-## Rate Limit Exceeded
+### Rate Limit Exceeded
 
 Pipedream limits the number of events that can be processed by a given interface (e.g. HTTP endpoints) during a given interval. This limit is most commonly reached for HTTP interfaces - see the [QPS limits documentation](/workflows/limits/#qps-queries-per-second) for more information on that limit.
 
 **This limit can be raised for HTTP endpoints**. [Reach out to our team](https://pipedream.com/support/) to request an increase.
 
-## Request Entity Too Large
+### Request Entity Too Large
 
 By default, Pipedream limits the size of incoming HTTP payloads. If you exceed this limit, you'll see a **Request Entity Too Large** error.
 
@@ -140,35 +175,35 @@ Pipedream supports two different ways to bypass this limit. Both of these interf
 - You can send large HTTP payloads by passing the `pipedream_upload_body=1` query string or an `x-pd-upload-body: 1` HTTP header in your HTTP request. [Read more here](/workflows/building-workflows/triggers/#sending-large-payloads).
 - You can upload multiple large files, like images and videos, using the [large file upload interface](/workflows/building-workflows/triggers/#large-file-support).
 
-## Function Payload Limit Exceeded
+### Function Payload Limit Exceeded
 
 The total size of `console.log()` statements, [step exports](/workflows/#step-exports), and the original event data sent to workflows and sources cannot exceed a combined size of {process.env.FUNCTION_PAYLOAD_LIMIT}. If you produce logs or step exports larger than this - for example, passing around large API responses, CSVs, or other data - you may encounter a **Function Payload Limit Exceeded** in your workflow.
 
 Often, this occurs when you pass large data between steps using [step exports](/workflows/#step-exports). You can avoid this error by [writing that data to the `/tmp` directory](/workflows/building-workflows/code/nodejs/working-with-files/#writing-a-file-to-tmp) in one step, and [reading the data into another step](/workflows/building-workflows/code/nodejs/working-with-files/#reading-a-file-from-tmp), which avoids the use of step exports and should keep you under the payload limit.
 
 Pipedream also compresses the function payload from your workflow, which can yield roughly a 2x-3x increase in payload size (somewhere between `12MB` and `18MB`), depending on the data.
 
-## JSON Nested Property Limit Exceeded
+### JSON Nested Property Limit Exceeded
 
 Working with nested JavaScript objects that have more than 256 nested objects will trigger a **JSON Nested Property Limit Exceeded** error.
 
 Often, objects with this many nested objects result from a programming error that explodes the object in an unexpected way. Please confirm the code you're using to convert data into an object is correctly parsing the object.
 
-## Event Queue Full
+### Event Queue Full
 
 Workflows have a maximum event queue size when using concurrency and throttling controls. If the number of unprocessed events exceeds the [maximum queue size](/workflows/building-workflows/settings/concurrency-and-throttling/#increasing-the-queue-size-for-a-workflow), you may encounter an **Event Queue Full** error.
 
 [Paid plans](https://pipedream.com/pricing) can [increase their queue size up to {process.env.MAX_WORKFLOW_QUEUE_SIZE}](/workflows/building-workflows/settings/concurrency-and-throttling/#increasing-the-queue-size-for-a-workflow) for a given workflow.
 
-## Credit Budget Exceeded
+### Credit Budget Exceeded
 
 Credit Budgets are configurable limits on your credit usage at the account or workspace level.
 
 If you're receiving this warning on a source or workflow, this means your allocated Credit Budget has been reached for the defined period.
 
 You can increase this limit at any time in the [billing area of your settings](https://pipedream.com/settings/billing).
 
-## Pipedream Internal Error
+### Pipedream Internal Error
 
 A `Pipedream Internal Error` is thrown whenever there's an exception during the building or executing of a workflow that's outside the scope of the code for the individual components (steps or actions).
 
@@ -220,20 +255,42 @@ Instead, use the specific package that exports the `pick` module alone:
 import pick from "lodash.pick"
 ```
 
-## Code was still running when the step ended
+## Is there a way to replay workflow events programmatically?
 
-This error occurs when Promises or asynchronous code is not properly finished before the next step begins execution.
+Not via the API, but you can bulk select and replay failed events using the [Event History](/workflows/event-history/).
 
-See the [Asynchronous section of the Node.js documentation](/workflows/building-workflows/code/nodejs/async/#the-problem) for more details.
+## How do I store and retrieve data across workflow executions?
 
-### How do I share my workflow with Support?
+If you operate your own database or data store, you can connect to it directly in Pipedream.
 
-First, navigate to your **Project Settings** and share your project with Pipedream Support.
+Pipedream also operates a [built-in key-value store](/workflows/data-management/data-stores/) that you can use to get and set data across workflow executions and different workflows.
 
-If your workflow is _not_ part of a Project, go to the **Workflow Settings** to grant access to us.
+## How do I delay the execution of a workflow?
 
-When filling out the request form at [https://pipedream.com/support](https://pipedream.com/support), please provide detailed information along with the URL from your browser's address bar, which should look something like:
+Use Pipedream's [built-in Delay actions](/workflows/building-workflows/control-flow/delay/) to delay a workflow at any step.
 
-```
-https://pipedream.com/@yourworkspace/workflows/projects/yourproject/test-workflow-pabc123
-```
+## How can my workflow run faster?
+
+Here are a few things that can help your workflow execute faster:
+
+1. **Increase memory:** Increase your [workflow memory](/workflows/building-workflows/settings/#memory) to at least 512 MB. Raising the memory limit will proportionally increase CPU resources, leading to improved performance and reduced latency.
+
+2. **Return static HTTP responses:** If your workflow is triggered by an HTTP source, return a [static HTTP response](/workflows/building-workflows/triggers/#http-responses) directly from the trigger configuration. This ensures the HTTP response is sent to the caller immediately, before the rest of the workflow steps are executed.
+
+3. **Simplify your workflow:** Reduce the number of [steps](/workflows/#code-actions) and [segments](/workflows/building-workflows/control-flow/#workflow-segments) in your workflow, combining multiple steps into one, if possible. This lowers the overhead involved in managing step execution and exports.
+
+4. **Activate warm workers:** Use [warm workers](/workflows/building-workflows/settings/#eliminate-cold-starts) to reduce the startup time of workflows. Set [as many warm workers](/workflows/building-workflows/settings/#how-many-workers-should-i-configure) as you want for high-volume traffic.
+
+## How can I save common functions as steps?
+
+You can create your own custom triggers and actions ("components") on Pipedream using [the Component API](/workflows/contributing/components/api/). These components are private to your account and can be used in any workflow.
+
+You can also publish common functions in your own package on a public registry like [npm](https://www.npmjs.com/) or [PyPI](https://pypi.org/).
+
+## Is Puppeteer supported in Pipedream?
+
+Yes, see [our Puppeteer docs](/workflows/building-workflows/code/nodejs/browser-automation/#puppeteer) for more detail.
+
+## Is Playwright supported in Pipedream?
+
+Yes, see [our Playwright docs](/workflows/building-workflows/code/nodejs/browser-automation/#playwright) for more detail.