You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat(cdk): add examples and improve descriptions in YAML schema
- Add GraphQL query examples for RequestBodyGraphQlQuery
- Enhance CustomIncrementalSync cursor_field with examples and title
- Fix stream name example format (string instead of array)
- Improve CursorPagination descriptions and add varied examples
- Enhance ApiKeyAuthenticator with more header examples and better descriptions
- Add realistic interpolation examples using {{ config['field'] }} syntax
- Focus on complex components where user input is not obvious
Co-Authored-By: AJ Steers <[email protected]>
Copy file name to clipboardExpand all lines: airbyte_cdk/sources/declarative/declarative_component_schema.yaml
+59-14Lines changed: 59 additions & 14 deletions
Original file line number
Diff line number
Diff line change
@@ -150,23 +150,30 @@ definitions:
150
150
enum: [ApiKeyAuthenticator]
151
151
api_token:
152
152
title: API Key
153
-
description: The API key to inject in the request. Fill it in the user inputs.
153
+
description: The API key token that will be injected into requests for authentication. This should reference a configuration field that contains the user's API key. Can include formatting like 'Bearer' or 'Token' prefixes.
154
154
type: string
155
155
interpolation_context:
156
156
- config
157
157
examples:
158
158
- "{{ config['api_key'] }}"
159
159
- "Token token={{ config['api_key'] }}"
160
+
- "Bearer {{ config['access_token'] }}"
161
+
- "{{ config['auth_token'] }}"
162
+
- "{{ config['secret_key'] }}"
160
163
header:
161
164
title: Header Name
162
-
description: The name of the HTTP header that will be set to the API key. This setting is deprecated, use inject_into instead. Header and inject_into can not be defined at the same time.
165
+
description: The name of the HTTP header where the API key will be injected. This setting is deprecated, use inject_into instead. Common headers include 'Authorization', 'X-API-Key', or custom headers specific to the API. Header and inject_into cannot be defined at the same time.
163
166
type: string
164
167
interpolation_context:
165
168
- config
166
169
examples:
167
170
- Authorization
168
171
- Api-Token
169
172
- X-Auth-Token
173
+
- X-API-Key
174
+
- X-RapidAPI-Key
175
+
- apikey
176
+
- token
170
177
inject_into:
171
178
title: Inject API Key Into Outgoing HTTP Request
172
179
description: Configure how the API Key will be sent in requests to the source API. Either inject_into or header has to be defined.
@@ -496,7 +503,7 @@ definitions:
496
503
enum: [CursorPagination]
497
504
cursor_value:
498
505
title: Cursor Value
499
-
description: Value of the cursor defining the next page to fetch.
506
+
description: Template string for the cursor value to be used in the next request. This should reference a field from the API response that indicates the next page position.
500
507
type: string
501
508
interpolation_context:
502
509
- config
@@ -508,12 +515,20 @@ definitions:
508
515
- "{{ headers.link.next.cursor }}"
509
516
- "{{ last_record['key'] }}"
510
517
- "{{ response['nextPage'] }}"
518
+
- "{{ response.next_cursor }}"
519
+
- "{{ last_record['id'] }}"
520
+
- "{{ headers['x-next-page'] }}"
521
+
- "{{ response.pagination.next_token }}"
511
522
page_size:
512
523
title: Page Size
513
-
description: The number of records to include in each pages.
524
+
description: The number of records to include in each page. This value will be used with cursor-based pagination to control the amount of data returned per request. Choose a value that balances API rate limits with sync performance.
514
525
type: integer
515
526
examples:
527
+
- 10
528
+
- 50
516
529
- 100
530
+
- 500
531
+
- 1000
517
532
stop_condition:
518
533
title: Stop Condition
519
534
description: Template string evaluating when to stop paginating.
description: The location of the value on a record that will be used as a bookmark during sync.
632
+
title: Cursor Field
633
+
description: The location of the value on a record that will be used as a bookmark during sync. This field should contain datetime values that increase over time to enable incremental syncing. The field must be at the top level of the record.
618
634
type: string
635
+
examples:
636
+
- "updated_at"
637
+
- "created_at"
638
+
- "last_modified"
639
+
- "timestamp"
640
+
- "{{ config['cursor_field'] }}"
619
641
$parameters:
620
642
type: object
621
643
additionalProperties: true
@@ -1496,11 +1518,16 @@ definitions:
1496
1518
enum: [DeclarativeStream]
1497
1519
name:
1498
1520
title: Name
1499
-
description: The stream name.
1521
+
description: The name of the stream as it will appear in the connector. This should be descriptive and match the data being synced (e.g., 'users', 'orders', 'transactions'). Use lowercase with underscores for consistency.
1500
1522
type: string
1501
1523
default: ""
1502
-
example:
1503
-
- "Users"
1524
+
examples:
1525
+
- "users"
1526
+
- "orders"
1527
+
- "transactions"
1528
+
- "events"
1529
+
- "user_profiles"
1530
+
- "order_items"
1504
1531
retriever:
1505
1532
title: Retriever
1506
1533
description: Component used to coordinate how records are extracted across stream slices and request pages.
@@ -3135,7 +3162,7 @@ definitions:
3135
3162
enum: [OffsetIncrement]
3136
3163
page_size:
3137
3164
title: Limit
3138
-
description: The number of records to include in each pages.
3165
+
description: The number of records to include in each page. This value will be used to increment the offset for subsequent requests. Choose a value that balances API rate limits with sync performance.
3139
3166
anyOf:
3140
3167
- type: integer
3141
3168
title: Number of Records
@@ -3145,7 +3172,11 @@ definitions:
3145
3172
- config
3146
3173
- response
3147
3174
examples:
3175
+
- 10
3176
+
- 50
3148
3177
- 100
3178
+
- 500
3179
+
- 1000
3149
3180
- "{{ config['page_size'] }}"
3150
3181
inject_on_first_request:
3151
3182
title: Inject Offset on First Request
@@ -3167,7 +3198,7 @@ definitions:
3167
3198
enum: [PageIncrement]
3168
3199
page_size:
3169
3200
title: Page Size
3170
-
description: The number of records to include in each pages.
3201
+
description: The number of records to include in each page. This value will be used to increment the page number for subsequent requests. Choose a value that balances API rate limits with sync performance.
3171
3202
interpolation_context:
3172
3203
- config
3173
3204
anyOf:
@@ -3176,7 +3207,11 @@ definitions:
3176
3207
- type: string
3177
3208
title: Interpolated Value
3178
3209
examples:
3210
+
- 10
3211
+
- 50
3179
3212
- 100
3213
+
- 500
3214
+
- 1000
3180
3215
- "100"
3181
3216
- "{{ config['page_size'] }}"
3182
3217
start_from_page:
@@ -3621,11 +3656,16 @@ definitions:
3621
3656
enum: [StateDelegatingStream]
3622
3657
name:
3623
3658
title: Name
3624
-
description: The stream name.
3659
+
description: The name of the stream as it will appear in the connector. This should be descriptive and match the data being synced (e.g., 'users', 'orders', 'transactions'). Use lowercase with underscores for consistency.
3625
3660
type: string
3626
3661
default: ""
3627
-
example:
3628
-
- "Users"
3662
+
examples:
3663
+
- "users"
3664
+
- "orders"
3665
+
- "transactions"
3666
+
- "events"
3667
+
- "user_profiles"
3668
+
- "order_items"
3629
3669
full_refresh_stream:
3630
3670
title: Full Refresh Stream
3631
3671
description: Component used to coordinate how records are extracted across stream slices and request pages when the state is empty or not provided.
@@ -4417,8 +4457,13 @@ definitions:
4417
4457
properties:
4418
4458
query:
4419
4459
type: string
4420
-
description: The GraphQL query to be executed
4460
+
description: The GraphQL query to be executed. This should be a valid GraphQL query string that will be sent in the request body.
Copy file name to clipboardExpand all lines: airbyte_cdk/sources/declarative/models/declarative_component_schema.py
+70-15Lines changed: 70 additions & 15 deletions
Original file line number
Diff line number
Diff line change
@@ -104,18 +104,22 @@ class CursorPagination(BaseModel):
104
104
type: Literal["CursorPagination"]
105
105
cursor_value: str=Field(
106
106
...,
107
-
description="Value of the cursor defining the next page to fetch.",
107
+
description="Template string for the cursor value to be used in the next request. This should reference a field from the API response that indicates the next page position.",
108
108
examples=[
109
109
"{{ headers.link.next.cursor }}",
110
110
"{{ last_record['key'] }}",
111
111
"{{ response['nextPage'] }}",
112
+
"{{ response.next_cursor }}",
113
+
"{{ last_record['id'] }}",
114
+
"{{ headers['x-next-page'] }}",
115
+
"{{ response.pagination.next_token }}",
112
116
],
113
117
title="Cursor Value",
114
118
)
115
119
page_size: Optional[int] =Field(
116
120
None,
117
-
description="The number of records to include in each pages.",
118
-
examples=[100],
121
+
description="The number of records to include in each page. This value will be used with cursor-based pagination to control the amount of data returned per request. Choose a value that balances API rate limits with sync performance.",
122
+
examples=[10, 50, 100, 500, 1000],
119
123
title="Page Size",
120
124
)
121
125
stop_condition: Optional[str] =Field(
@@ -185,7 +189,15 @@ class Config:
185
189
)
186
190
cursor_field: str=Field(
187
191
...,
188
-
description="The location of the value on a record that will be used as a bookmark during sync.",
192
+
description="The location of the value on a record that will be used as a bookmark during sync. This field should contain datetime values that increase over time to enable incremental syncing. The field must be at the top level of the record.",
@@ -1201,8 +1213,8 @@ class OffsetIncrement(BaseModel):
1201
1213
type: Literal["OffsetIncrement"]
1202
1214
page_size: Optional[Union[int, str]] =Field(
1203
1215
None,
1204
-
description="The number of records to include in each pages.",
1205
-
examples=[100, "{{ config['page_size'] }}"],
1216
+
description="The number of records to include in each page. This value will be used to increment the offset for subsequent requests. Choose a value that balances API rate limits with sync performance.",
description="The number of records to include in each page. This value will be used to increment the page number for subsequent requests. Choose a value that balances API rate limits with sync performance.",
description="The API key token that will be injected into requests for authentication. This should reference a configuration field that contains the user's API key. Can include formatting like 'Bearer' or 'Token' prefixes.",
1757
+
examples=[
1758
+
"{{ config['api_key'] }}",
1759
+
"Token token={{ config['api_key'] }}",
1760
+
"Bearer {{ config['access_token'] }}",
1761
+
"{{ config['auth_token'] }}",
1762
+
"{{ config['secret_key'] }}",
1763
+
],
1737
1764
title="API Key",
1738
1765
)
1739
1766
header: Optional[str] =Field(
1740
1767
None,
1741
-
description="The name of the HTTP header that will be set to the API key. This setting is deprecated, use inject_into instead. Header and inject_into can not be defined at the same time.",
description="The name of the HTTP header where the API key will be injected. This setting is deprecated, use inject_into instead. Common headers include 'Authorization', 'X-API-Key', or custom headers specific to the API. Header and inject_into cannot be defined at the same time.",
description="The name of the stream as it will appear in the connector. This should be descriptive and match the data being synced (e.g., 'users', 'orders', 'transactions'). Use lowercase with underscores for consistency.",
description="The name of the stream as it will appear in the connector. This should be descriptive and match the data being synced (e.g., 'users', 'orders', 'transactions'). Use lowercase with underscores for consistency.",
0 commit comments