Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/community/contributing-to-airbyte/writing-docs.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,13 +226,15 @@ To enable badges, include `products` in the Markdown metadata. The following val
- `oss-community`: Core only
- `oss-enterprise`: Self-Managed Enterprise only
- `cloud`: Standard and Plus (also enables Pro and Enterprise Flex due to Cloud tier inheritance)
- `cloud-plus`: Plus only (also enables Pro and Enterprise Flex due to Cloud tier inheritance)
- `cloud-teams`: Pro only (also enables Enterprise Flex due to Cloud tier inheritance)
- `enterprise-flex`: Enterprise Flex only
- `embedded`: Embedded only (hidden if not specified - there is no off state for the Embedded badge)

**Cloud tier inheritance:** higher Cloud plans automatically inherit availability from lower tiers:

- If you specify `cloud`: Standard, Plus, Pro, and Enterprise Flex badges all become enabled
- If you specify `cloud-plus`: Plus, Pro, and Enterprise Flex badges become enabled - Standard turns off
- If you specify `cloud-teams`: Pro and Enterprise Flex badges become enabled - Standard and Plus turn off
- If you specify `enterprise-flex`: Only Enterprise Flex badge becomes enabled

Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
55 changes: 55 additions & 0 deletions docs/platform/cloud/managing-airbyte-cloud/manage-data-workers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
---
products: cloud-plus
---

# Monitor data worker usage

If you are on a capacity-based plan, you can monitor your data worker usage across your organization. To view data worker usage, you need the **organization admin** role.

## Open the usage chart

From the navigation bar, click **Organization settings** > **Usage**.

## How to interpret the chart

The chart shows daily maximum concurrent data worker usage, from all workspaces in a region, over a period of time. Each bar represents one day. The chart stacks all workspaces in that region so you can see which workspace uses the most data workers each day.

![Page showing a region, a period of time, and a bar chart with maximum daily data worker usage within that period of time and region](assets/data-worker-usage.png)

Hover on a day to see more details about it.

## Filter the chart

- To change the region, click the region dropdown in the chart's upper left corner and choose a different region.

- To change the date range, click the date dropdown in the chart's upper right corner and choose a new date range.

## What to do if you hit your data worker limit

An infrequent instance of maximum usage probably isn't a problem. If you're regularly hitting your data worker limit, you have three options.

- Accept that Airbyte may queue or throttle your connections.

- Reschedule some connections so they run at different times of the day, week, or month.

- Buy more data workers to increase capacity.

### Optimize data worker usage

If you can, it's preferable to optimize Airbyte by rescheduling connections outside of busy periods.

- **If your usage has peaks and valleys**, find connections that run on busy days and move them to lower-usage days.

- **If your usage looks consistently high**, examine your scheduling patterns within a day. If a large number of connections start at the same time, data worker usage spikes.

- Stagger start times over a longer period to allow some connections to finish before others begin.

- Avoid starting all your syncs at the top of the hour. Starting them at :15, :30, and :45 can more evenly distribute work.

- If a large number of connections run overnight, data workers might look fully utilized, but sit unused during daylight hours.

- **If sandbox/staging workspaces consume too much capacity**, consider reducing the frequency of syncs in less critical workspaces.

### Buy more data workers

If you've tried to optimize scheduling and still need more data workers, contact your Airbyte representative or [talk to sales](https://www.airbyte.com/talk-to-sales).
1 change: 1 addition & 0 deletions docusaurus/sidebar-platform.js
Original file line number Diff line number Diff line change
Expand Up @@ -382,6 +382,7 @@ module.exports = {
items: [
"organizations-workspaces/organizations/switch-organizations",
"cloud/managing-airbyte-cloud/manage-credits",
"cloud/managing-airbyte-cloud/manage-data-workers",
],
},
{
Expand Down
7 changes: 4 additions & 3 deletions docusaurus/src/components/ProductInformation.jsx
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ export const ProductInformation = ({ products }) => {
products["oss-enterprise"] || products["oss-*"] || products["all"];
const cloud = products["cloud"] || products["cloud-teams"] || products["all"];
// cloud add-ons need to be specifically marked and are not part of the "all" shorthand
const cloudPlus = products["cloud-plus"];
const cloudTeams = products["cloud-teams"];
const enterpriseFlex = products["enterprise-flex"];
const embedded = products["embedded"];
Expand All @@ -53,11 +54,11 @@ export const ProductInformation = ({ products }) => {
>
Standard
</Badge>
<Badge available={cloud && !cloudTeams && !enterpriseFlex}>Plus</Badge>
<Badge available={cloud || cloudTeams} title="Formerly Cloud Teams">
<Badge available={cloud && !cloudTeams && !enterpriseFlex || cloudPlus}>Plus</Badge>
<Badge available={cloudPlus || cloud || cloudTeams} title="Formerly Cloud Teams">
Pro
</Badge>
<Badge available={cloud || cloudTeams || enterpriseFlex}>
<Badge available={cloudPlus || cloud || cloudTeams || enterpriseFlex}>
Enterprise Flex
</Badge>
<Badge available={ossEnterprise}>Self-Managed Enterprise</Badge>
Expand Down
Loading