Connector Builder: Snapchat Marketing incremental sync error `PerPartitionStreamSlice is not JSON serializable` #31274

Abuton · 2023-10-11T11:39:10Z

Abuton
Oct 11, 2023

Hello,

I'm trying to build the Snapchat marketing source connector using the builder option on Airbyte. I have implemented some streams like organization, adaccount, campaigns and ads. Now I'm currently faced with the challenge of adding incremental sync to the stream. I was able to add the incremental sync option to adaccount but when I tried to do the same for campaigns and ads streams it failed with the following error:
Object of type PerPartitionStreamSlice is not JSON serializable

As anyone faced this error before or has a suggestion as to how I can resolve it?

I'm pasting the generated yaml file below:

version: 0.50.2
type: DeclarativeSource
check:
  type: CheckStream
  stream_names:
    - Organization
streams:
  - type: DeclarativeStream
    name: Organization
    primary_key:
      - id
    schema_loader:
      type: InlineSchemaLoader
      schema:
        $schema: http://json-schema.org/schema#
        properties:
          contact_name:
            type: string
          country:
            type: string
          created_at:
            type: string
          id:
            type: string
          locality:
            type: string
          name:
            type: string
          organization:
            properties:
              accepted_term_version:
                type: string
              address_line_1:
                type: string
              administrative_district_level_1:
                type: string
              business_type:
                type: string
              configuration_settings:
                properties:
                  notifications_enabled:
                    type: boolean
                type: object
              contact_email:
                type: string
              contact_name:
                type: string
              contact_phone:
                type: string
              country:
                type: string
              createdByCaller:
                type: boolean
              created_at:
                type: string
              id:
                type: string
              is_agency:
                type: boolean
              locality:
                type: string
              marketing_optin:
                type: boolean
              my_display_name:
                type: string
              my_invited_email:
                type: string
              my_member_id:
                type: string
              name:
                type: string
              postal_code:
                type: string
              roles:
                items:
                  type: string
                type: array
              state:
                type: string
              type:
                type: string
              updated_at:
                type: string
            type: object
          roles:
            items:
              type: string
            type: array
          updated_at:
            type: string
        type: object
    retriever:
      type: SimpleRetriever
      requester:
        type: HttpRequester
        url_base: https://adsapi.snapchat.com/v1/
        path: me/organizations
        http_method: GET
        request_parameters: {}
        request_headers: {}
        authenticator:
          type: OAuthAuthenticator
          client_id: '{{ config[''client_id''] }}'
          grant_type: refresh_token
          client_secret: '{{ config[''client_secret''] }}'
          refresh_token: '{{ config[''client_refresh_token''] }}'
          refresh_request_body: {}
          token_refresh_endpoint: https://accounts.snapchat.com/login/oauth2/access_token
        request_body_json: {}
      record_selector:
        type: RecordSelector
        extractor:
          type: DpathExtractor
          field_path:
            - organizations
      paginator:
        type: NoPagination
    transformations:
      - type: RemoveFields
        field_pointers:
          - - sub_request_status
      - type: AddFields
        fields:
          - path:
              - id
            value: '{{ record.organization.id }}'
      - type: AddFields
        fields:
          - path:
              - updated_at
            value: '{{ record.organization.updated_at }}'
      - type: AddFields
        fields:
          - path:
              - created_at
            value: '{{ record.organization.created_at }}'
      - type: AddFields
        fields:
          - path:
              - country
            value: '{{ record.organization.country }}'
      - type: AddFields
        fields:
          - path:
              - name
            value: '{{ record.organization.name }}'
      - type: AddFields
        fields:
          - path:
              - contact_name
            value: '{{ record.organization.contact_name }}'
      - type: AddFields
        fields:
          - path:
              - locality
            value: '{{ record.organization.locality }}'
      - type: AddFields
        fields:
          - path:
              - roles
            value: '{{ record.organization.roles }}'
    incremental_sync:
      type: DatetimeBasedCursor
      cursor_field: updated_at
      lookback_window: P28D
      cursor_datetime_formats:
        - '%Y-%m-%dT%H:%M:%S.%fZ'
      datetime_format: '%Y-%m-%dT%H:%M:%S.%fZ'
      start_datetime:
        type: MinMaxDatetime
        datetime: '{{ config[''start_date''] }}'
        datetime_format: '%Y-%m-%d'
      end_datetime:
        type: MinMaxDatetime
        datetime: '{{ now_utc().strftime(''%Y-%m-%dT%H:%M:%SZ'') }}'
        datetime_format: '%Y-%m-%dT%H:%M:%SZ'
  - type: DeclarativeStream
    name: Adaccounts
    primary_key:
      - ad_account_id
    schema_loader:
      type: InlineSchemaLoader
      schema:
        $schema: http://json-schema.org/schema#
        properties:
          ad_account_id:
            type: string
          adaccount:
            properties:
              advertiser_organization_id:
                type: string
              agency_client_metadata:
                type: object
              agency_representing_client:
                type: boolean
              billing_center_id:
                type: string
              billing_type:
                type: string
              cell_ids:
                items:
                  type: string
                type: array
              client_paying_invoices:
                type: boolean
              created_at:
                type: string
              currency:
                type: string
              funding_source_ids:
                items:
                  type: string
                type: array
              id:
                type: string
              name:
                type: string
              organization_id:
                type: string
              regulations:
                properties:
                  restricted_delivery_signals:
                    type: boolean
                type: object
              status:
                type: string
              timezone:
                type: string
              type:
                type: string
              updated_at:
                type: string
            type: object
          created_at:
            type: string
          name:
            type: string
          organization_id:
            type: string
          status:
            type: string
          type:
            type: string
          updated_at:
            type: string
        type: object
    retriever:
      type: SimpleRetriever
      requester:
        type: HttpRequester
        url_base: https://adsapi.snapchat.com/v1/
        path: organizations/{{ stream_partition.id }}/adaccounts
        http_method: GET
        request_parameters: {}
        request_headers: {}
        authenticator:
          type: OAuthAuthenticator
          client_id: '{{ config[''client_id''] }}'
          grant_type: refresh_token
          client_secret: '{{ config[''client_secret''] }}'
          refresh_token: '{{ config[''client_refresh_token''] }}'
          refresh_request_body: {}
          token_refresh_endpoint: https://accounts.snapchat.com/login/oauth2/access_token
        error_handler:
          type: CompositeErrorHandler
          error_handlers:
            - type: DefaultErrorHandler
              backoff_strategies:
                - type: ConstantBackoffStrategy
                  backoff_time_in_seconds: 5
              response_filters:
                - type: HttpResponseFilter
                  action: RETRY
                  http_codes:
                    - 429
                    - 500
                  error_message: Too Many Request
        request_body_json: {}
      record_selector:
        type: RecordSelector
        extractor:
          type: DpathExtractor
          field_path:
            - adaccounts
      paginator:
        type: DefaultPaginator
        page_token_option:
          type: RequestOption
          inject_into: request_parameter
          field_name: offset
        page_size_option:
          type: RequestOption
          field_name: limit
          inject_into: request_parameter
        pagination_strategy:
          type: OffsetIncrement
          page_size: 200
      partition_router:
        - type: SubstreamPartitionRouter
          parent_stream_configs:
            - type: ParentStreamConfig
              parent_key: id
              partition_field: id
              stream:
                type: DeclarativeStream
                name: Organization
                primary_key:
                  - id
                schema_loader:
                  type: InlineSchemaLoader
                  schema:
                    $schema: http://json-schema.org/schema#
                    properties:
                      contact_name:
                        type: string
                      country:
                        type: string
                      created_at:
                        type: string
                      id:
                        type: string
                      locality:
                        type: string
                      name:
                        type: string
                      organization:
                        properties:
                          accepted_term_version:
                            type: string
                          address_line_1:
                            type: string
                          administrative_district_level_1:
                            type: string
                          business_type:
                            type: string
                          configuration_settings:
                            properties:
                              notifications_enabled:
                                type: boolean
                            type: object
                          contact_email:
                            type: string
                          contact_name:
                            type: string
                          contact_phone:
                            type: string
                          country:
                            type: string
                          createdByCaller:
                            type: boolean
                          created_at:
                            type: string
                          id:
                            type: string
                          is_agency:
                            type: boolean
                          locality:
                            type: string
                          marketing_optin:
                            type: boolean
                          my_display_name:
                            type: string
                          my_invited_email:
                            type: string
                          my_member_id:
                            type: string
                          name:
                            type: string
                          postal_code:
                            type: string
                          roles:
                            items:
                              type: string
                            type: array
                          state:
                            type: string
                          type:
                            type: string
                          updated_at:
                            type: string
                        type: object
                      roles:
                        items:
                          type: string
                        type: array
                      updated_at:
                        type: string
                    type: object
                retriever:
                  type: SimpleRetriever
                  requester:
                    type: HttpRequester
                    url_base: https://adsapi.snapchat.com/v1/
                    path: me/organizations
                    http_method: GET
                    request_parameters: {}
                    request_headers: {}
                    authenticator:
                      type: OAuthAuthenticator
                      client_id: '{{ config[''client_id''] }}'
                      grant_type: refresh_token
                      client_secret: '{{ config[''client_secret''] }}'
                      refresh_token: '{{ config[''client_refresh_token''] }}'
                      refresh_request_body: {}
                      token_refresh_endpoint: https://accounts.snapchat.com/login/oauth2/access_token
                    request_body_json: {}
                  record_selector:
                    type: RecordSelector
                    extractor:
                      type: DpathExtractor
                      field_path:
                        - organizations
                  paginator:
                    type: NoPagination
                transformations:
                  - type: RemoveFields
                    field_pointers:
                      - - sub_request_status
                  - type: AddFields
                    fields:
                      - path:
                          - id
                        value: '{{ record.organization.id }}'
                  - type: AddFields
                    fields:
                      - path:
                          - updated_at
                        value: '{{ record.organization.updated_at }}'
                  - type: AddFields
                    fields:
                      - path:
                          - created_at
                        value: '{{ record.organization.created_at }}'
                  - type: AddFields
                    fields:
                      - path:
                          - country
                        value: '{{ record.organization.country }}'
                  - type: AddFields
                    fields:
                      - path:
                          - name
                        value: '{{ record.organization.name }}'
                  - type: AddFields
                    fields:
                      - path:
                          - contact_name
                        value: '{{ record.organization.contact_name }}'
                  - type: AddFields
                    fields:
                      - path:
                          - locality
                        value: '{{ record.organization.locality }}'
                  - type: AddFields
                    fields:
                      - path:
                          - roles
                        value: '{{ record.organization.roles }}'
                incremental_sync:
                  type: DatetimeBasedCursor
                  cursor_field: updated_at
                  lookback_window: P28D
                  cursor_datetime_formats:
                    - '%Y-%m-%dT%H:%M:%S.%fZ'
                  datetime_format: '%Y-%m-%dT%H:%M:%S.%fZ'
                  start_datetime:
                    type: MinMaxDatetime
                    datetime: '{{ config[''start_date''] }}'
                    datetime_format: '%Y-%m-%d'
                  end_datetime:
                    type: MinMaxDatetime
                    datetime: '{{ now_utc().strftime(''%Y-%m-%dT%H:%M:%SZ'') }}'
                    datetime_format: '%Y-%m-%dT%H:%M:%SZ'
    transformations:
      - type: RemoveFields
        field_pointers:
          - - sub_request_status
      - type: AddFields
        fields:
          - path:
              - ad_account_id
            value: '{{ record.adaccount.id }}'
      - type: AddFields
        fields:
          - path:
              - name
            value: '{{ record.adaccount.name }}'
      - type: AddFields
        fields:
          - path:
              - type
            value: '{{ record.adaccount.type }}'
      - type: AddFields
        fields:
          - path:
              - status
            value: '{{ record.adaccount.status }}'
      - type: AddFields
        fields:
          - path:
              - updated_at
            value: '{{ record.adaccount.updated_at }}'
      - type: AddFields
        fields:
          - path:
              - created_at
            value: '{{ record.adaccount.created_at }}'
      - type: AddFields
        fields:
          - path:
              - organization_id
            value: '{{ record.adaccount.organization_id }}'
    incremental_sync:
      type: DatetimeBasedCursor
      cursor_field: updated_at
      cursor_datetime_formats:
        - '%Y-%m-%dT%H:%M:%S.%fZ'
      datetime_format: '%Y-%m-%dT%H:%M:%S.%fZ'
      start_datetime:
        type: MinMaxDatetime
        datetime: '{{ config[''start_date''] }}'
        datetime_format: '%Y-%m-%d'
      end_datetime:
        type: MinMaxDatetime
        datetime: '{{ now_utc().strftime(''%Y-%m-%dT%H:%M:%SZ'') }}'
        datetime_format: '%Y-%m-%dT%H:%M:%SZ'
  - type: DeclarativeStream
    name: Campaigns
    primary_key:
      - id
    schema_loader:
      type: InlineSchemaLoader
      schema:
        $schema: http://json-schema.org/schema#
        properties:
          ad_account_id:
            type: string
          campaign:
            properties:
              ad_account_id:
                type: string
              buy_model:
                type: string
              created_at:
                type: string
              creation_state:
                type: string
              delivery_status:
                items:
                  type: string
                type: array
              end_time:
                type: string
              id:
                type: string
              lifetime_spend_cap_micro:
                type: number
              name:
                type: string
              objective:
                type: string
              start_time:
                type: string
              status:
                type: string
              updated_at:
                type: string
            type: object
          created_at:
            type: string
          id:
            type: string
          name:
            type: string
          objective:
            type: string
          start_time:
            type: string
          status:
            type: string
          updated_at:
            type: string
        type: object
    retriever:
      type: SimpleRetriever
      requester:
        type: HttpRequester
        url_base: https://adsapi.snapchat.com/v1/
        path: adaccounts/{{ stream_partition.id }}/campaigns
        http_method: GET
        request_parameters: {}
        request_headers: {}
        authenticator:
          type: OAuthAuthenticator
          client_id: '{{ config[''client_id''] }}'
          grant_type: refresh_token
          client_secret: '{{ config[''client_secret''] }}'
          refresh_token: '{{ config[''client_refresh_token''] }}'
          refresh_request_body: {}
          token_refresh_endpoint: https://accounts.snapchat.com/login/oauth2/access_token
        request_body_json: {}
      record_selector:
        type: RecordSelector
        extractor:
          type: DpathExtractor
          field_path:
            - campaigns
      paginator:
        type: DefaultPaginator
        page_token_option:
          type: RequestOption
          inject_into: request_parameter
          field_name: offset
        page_size_option:
          type: RequestOption
          field_name: limit
          inject_into: request_parameter
        pagination_strategy:
          type: OffsetIncrement
          page_size: 200
      partition_router:
        - type: SubstreamPartitionRouter
          parent_stream_configs:
            - type: ParentStreamConfig
              parent_key: ad_account_id
              partition_field: id
              stream:
                type: DeclarativeStream
                name: Adaccounts
                primary_key:
                  - ad_account_id
                schema_loader:
                  type: InlineSchemaLoader
                  schema:
                    $schema: http://json-schema.org/schema#
                    properties:
                      ad_account_id:
                        type: string
                      adaccount:
                        properties:
                          advertiser_organization_id:
                            type: string
                          agency_client_metadata:
                            type: object
                          agency_representing_client:
                            type: boolean
                          billing_center_id:
                            type: string
                          billing_type:
                            type: string
                          cell_ids:
                            items:
                              type: string
                            type: array
                          client_paying_invoices:
                            type: boolean
                          created_at:
                            type: string
                          currency:
                            type: string
                          funding_source_ids:
                            items:
                              type: string
                            type: array
                          id:
                            type: string
                          name:
                            type: string
                          organization_id:
                            type: string
                          regulations:
                            properties:
                              restricted_delivery_signals:
                                type: boolean
                            type: object
                          status:
                            type: string
                          timezone:
                            type: string
                          type:
                            type: string
                          updated_at:
                            type: string
                        type: object
                      created_at:
                        type: string
                      name:
                        type: string
                      organization_id:
                        type: string
                      status:
                        type: string
                      type:
                        type: string
                      updated_at:
                        type: string
                    type: object
                retriever:
                  type: SimpleRetriever
                  requester:
                    type: HttpRequester
                    url_base: https://adsapi.snapchat.com/v1/
                    path: organizations/{{ stream_partition.id }}/adaccounts
                    http_method: GET
                    request_parameters: {}
                    request_headers: {}
                    authenticator:
                      type: OAuthAuthenticator
                      client_id: '{{ config[''client_id''] }}'
                      grant_type: refresh_token
                      client_secret: '{{ config[''client_secret''] }}'
                      refresh_token: '{{ config[''client_refresh_token''] }}'
                      refresh_request_body: {}
                      token_refresh_endpoint: https://accounts.snapchat.com/login/oauth2/access_token
                    error_handler:
                      type: CompositeErrorHandler
                      error_handlers:
                        - type: DefaultErrorHandler
                          backoff_strategies:
                            - type: ConstantBackoffStrategy
                              backoff_time_in_seconds: 5
                          response_filters:
                            - type: HttpResponseFilter
                              action: RETRY
                              http_codes:
                                - 429
                                - 500
                              error_message: Too Many Request
                    request_body_json: {}
                  record_selector:
                    type: RecordSelector
                    extractor:
                      type: DpathExtractor
                      field_path:
                        - adaccounts
                  paginator:
                    type: DefaultPaginator
                    page_token_option:
                      type: RequestOption
                      inject_into: request_parameter
                      field_name: offset
                    page_size_option:
                      type: RequestOption
                      field_name: limit
                      inject_into: request_parameter
                    pagination_strategy:
                      type: OffsetIncrement
                      page_size: 200
                  partition_router:
                    - type: SubstreamPartitionRouter
                      parent_stream_configs:
                        - type: ParentStreamConfig
                          parent_key: id
                          partition_field: id
                          stream:
                            type: DeclarativeStream
                            name: Organization
                            primary_key:
                              - id
                            schema_loader:
                              type: InlineSchemaLoader
                              schema:
                                $schema: http://json-schema.org/schema#
                                properties:
                                  contact_name:
                                    type: string
                                  country:
                                    type: string
                                  created_at:
                                    type: string
                                  id:
                                    type: string
                                  locality:
                                    type: string
                                  name:
                                    type: string
                                  organization:
                                    properties:
                                      accepted_term_version:
                                        type: string
                                      address_line_1:
                                        type: string
                                      administrative_district_level_1:
                                        type: string
                                      business_type:
                                        type: string
                                      configuration_settings:
                                        properties:
                                          notifications_enabled:
                                            type: boolean
                                        type: object
                                      contact_email:
                                        type: string
                                      contact_name:
                                        type: string
                                      contact_phone:
                                        type: string
                                      country:
                                        type: string
                                      createdByCaller:
                                        type: boolean
                                      created_at:
                                        type: string
                                      id:
                                        type: string
                                      is_agency:
                                        type: boolean
                                      locality:
                                        type: string
                                      marketing_optin:
                                        type: boolean
                                      my_display_name:
                                        type: string
                                      my_invited_email:
                                        type: string
                                      my_member_id:
                                        type: string
                                      name:
                                        type: string
                                      postal_code:
                                        type: string
                                      roles:
                                        items:
                                          type: string
                                        type: array
                                      state:
                                        type: string
                                      type:
                                        type: string
                                      updated_at:
                                        type: string
                                    type: object
                                  roles:
                                    items:
                                      type: string
                                    type: array
                                  updated_at:
                                    type: string
                                type: object
                            retriever:
                              type: SimpleRetriever
                              requester:
                                type: HttpRequester
                                url_base: https://adsapi.snapchat.com/v1/
                                path: me/organizations
                                http_method: GET
                                request_parameters: {}
                                request_headers: {}
                                authenticator:
                                  type: OAuthAuthenticator
                                  client_id: '{{ config[''client_id''] }}'
                                  grant_type: refresh_token
                                  client_secret: '{{ config[''client_secret''] }}'
                                  refresh_token: '{{ config[''client_refresh_token''] }}'
                                  refresh_request_body: {}
                                  token_refresh_endpoint: >-
                                    https://accounts.snapchat.com/login/oauth2/access_token
                                request_body_json: {}
                              record_selector:
                                type: RecordSelector
                                extractor:
                                  type: DpathExtractor
                                  field_path:
                                    - organizations
                              paginator:
                                type: NoPagination
                            transformations:
                              - type: RemoveFields
                                field_pointers:
                                  - - sub_request_status
                              - type: AddFields
                                fields:
                                  - path:
                                      - id
                                    value: '{{ record.organization.id }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - updated_at
                                    value: '{{ record.organization.updated_at }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - created_at
                                    value: '{{ record.organization.created_at }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - country
                                    value: '{{ record.organization.country }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - name
                                    value: '{{ record.organization.name }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - contact_name
                                    value: '{{ record.organization.contact_name }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - locality
                                    value: '{{ record.organization.locality }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - roles
                                    value: '{{ record.organization.roles }}'
                            incremental_sync:
                              type: DatetimeBasedCursor
                              cursor_field: updated_at
                              lookback_window: P28D
                              cursor_datetime_formats:
                                - '%Y-%m-%dT%H:%M:%S.%fZ'
                              datetime_format: '%Y-%m-%dT%H:%M:%S.%fZ'
                              start_datetime:
                                type: MinMaxDatetime
                                datetime: '{{ config[''start_date''] }}'
                                datetime_format: '%Y-%m-%d'
                              end_datetime:
                                type: MinMaxDatetime
                                datetime: '{{ now_utc().strftime(''%Y-%m-%dT%H:%M:%SZ'') }}'
                                datetime_format: '%Y-%m-%dT%H:%M:%SZ'
                transformations:
                  - type: RemoveFields
                    field_pointers:
                      - - sub_request_status
                  - type: AddFields
                    fields:
                      - path:
                          - ad_account_id
                        value: '{{ record.adaccount.id }}'
                  - type: AddFields
                    fields:
                      - path:
                          - name
                        value: '{{ record.adaccount.name }}'
                  - type: AddFields
                    fields:
                      - path:
                          - type
                        value: '{{ record.adaccount.type }}'
                  - type: AddFields
                    fields:
                      - path:
                          - status
                        value: '{{ record.adaccount.status }}'
                  - type: AddFields
                    fields:
                      - path:
                          - updated_at
                        value: '{{ record.adaccount.updated_at }}'
                  - type: AddFields
                    fields:
                      - path:
                          - created_at
                        value: '{{ record.adaccount.created_at }}'
                  - type: AddFields
                    fields:
                      - path:
                          - organization_id
                        value: '{{ record.adaccount.organization_id }}'
                incremental_sync:
                  type: DatetimeBasedCursor
                  cursor_field: updated_at
                  cursor_datetime_formats:
                    - '%Y-%m-%dT%H:%M:%S.%fZ'
                  datetime_format: '%Y-%m-%dT%H:%M:%S.%fZ'
                  start_datetime:
                    type: MinMaxDatetime
                    datetime: '{{ config[''start_date''] }}'
                    datetime_format: '%Y-%m-%d'
                  end_datetime:
                    type: MinMaxDatetime
                    datetime: '{{ now_utc().strftime(''%Y-%m-%dT%H:%M:%SZ'') }}'
                    datetime_format: '%Y-%m-%dT%H:%M:%SZ'
    transformations:
      - type: RemoveFields
        field_pointers:
          - - sub_request_status
      - type: AddFields
        fields:
          - path:
              - id
            value: '{{ record.campaign.id }}'
      - type: AddFields
        fields:
          - path:
              - updated_at
            value: '{{ record.campaign.updated_at }}'
      - type: AddFields
        fields:
          - path:
              - created_at
            value: '{{ record.campaign.created_at }}'
      - type: AddFields
        fields:
          - path:
              - name
            value: '{{ record.campaign.name }}'
      - type: AddFields
        fields:
          - path:
              - ad_account_id
            value: '{{ record.campaign.ad_account_id }}'
      - type: AddFields
        fields:
          - path:
              - status
            value: '{{ record.campaign.status }}'
      - type: AddFields
        fields:
          - path:
              - objective
            value: '{{ record.campaign.objective }}'
      - type: AddFields
        fields:
          - path:
              - start_time
            value: '{{ record.campaign.start_time }}'
  - type: DeclarativeStream
    name: Ads
    primary_key:
      - id
    schema_loader:
      type: InlineSchemaLoader
      schema:
        $schema: http://json-schema.org/schema#
        properties:
          ad:
            properties:
              ad_squad_id:
                type: string
              created_at:
                type: string
              creative_id:
                type: string
              delivery_status:
                items:
                  type: string
                type: array
              id:
                type: string
              name:
                type: string
              render_type:
                type: string
              review_status:
                type: string
              status:
                type: string
              type:
                type: string
              updated_at:
                type: string
            type: object
          ad_squad_id:
            type: string
          created_at:
            type: string
          creative_id:
            type: string
          id:
            type: string
          name:
            type: string
          status:
            type: string
          updated_at:
            type: string
        type: object
    retriever:
      type: SimpleRetriever
      requester:
        type: HttpRequester
        url_base: https://adsapi.snapchat.com/v1/
        path: adaccounts/{{ stream_partition.id }}/ads
        http_method: GET
        request_parameters: {}
        request_headers: {}
        authenticator:
          type: OAuthAuthenticator
          client_id: '{{ config[''client_id''] }}'
          grant_type: refresh_token
          client_secret: '{{ config[''client_secret''] }}'
          refresh_token: '{{ config[''client_refresh_token''] }}'
          refresh_request_body: {}
          token_refresh_endpoint: https://accounts.snapchat.com/login/oauth2/access_token
        request_body_json: {}
      record_selector:
        type: RecordSelector
        extractor:
          type: DpathExtractor
          field_path:
            - ads
      paginator:
        type: DefaultPaginator
        page_size_option:
          type: RequestOption
          field_name: limit
          inject_into: request_parameter
        pagination_strategy:
          type: OffsetIncrement
          page_size: 200
      partition_router:
        - type: SubstreamPartitionRouter
          parent_stream_configs:
            - type: ParentStreamConfig
              parent_key: ad_account_id
              partition_field: id
              stream:
                type: DeclarativeStream
                name: Adaccounts
                primary_key:
                  - ad_account_id
                schema_loader:
                  type: InlineSchemaLoader
                  schema:
                    $schema: http://json-schema.org/schema#
                    properties:
                      ad_account_id:
                        type: string
                      adaccount:
                        properties:
                          advertiser_organization_id:
                            type: string
                          agency_client_metadata:
                            type: object
                          agency_representing_client:
                            type: boolean
                          billing_center_id:
                            type: string
                          billing_type:
                            type: string
                          cell_ids:
                            items:
                              type: string
                            type: array
                          client_paying_invoices:
                            type: boolean
                          created_at:
                            type: string
                          currency:
                            type: string
                          funding_source_ids:
                            items:
                              type: string
                            type: array
                          id:
                            type: string
                          name:
                            type: string
                          organization_id:
                            type: string
                          regulations:
                            properties:
                              restricted_delivery_signals:
                                type: boolean
                            type: object
                          status:
                            type: string
                          timezone:
                            type: string
                          type:
                            type: string
                          updated_at:
                            type: string
                        type: object
                      created_at:
                        type: string
                      name:
                        type: string
                      organization_id:
                        type: string
                      status:
                        type: string
                      type:
                        type: string
                      updated_at:
                        type: string
                    type: object
                retriever:
                  type: SimpleRetriever
                  requester:
                    type: HttpRequester
                    url_base: https://adsapi.snapchat.com/v1/
                    path: organizations/{{ stream_partition.id }}/adaccounts
                    http_method: GET
                    request_parameters: {}
                    request_headers: {}
                    authenticator:
                      type: OAuthAuthenticator
                      client_id: '{{ config[''client_id''] }}'
                      grant_type: refresh_token
                      client_secret: '{{ config[''client_secret''] }}'
                      refresh_token: '{{ config[''client_refresh_token''] }}'
                      refresh_request_body: {}
                      token_refresh_endpoint: https://accounts.snapchat.com/login/oauth2/access_token
                    error_handler:
                      type: CompositeErrorHandler
                      error_handlers:
                        - type: DefaultErrorHandler
                          backoff_strategies:
                            - type: ConstantBackoffStrategy
                              backoff_time_in_seconds: 5
                          response_filters:
                            - type: HttpResponseFilter
                              action: RETRY
                              http_codes:
                                - 429
                                - 500
                              error_message: Too Many Request
                    request_body_json: {}
                  record_selector:
                    type: RecordSelector
                    extractor:
                      type: DpathExtractor
                      field_path:
                        - adaccounts
                  paginator:
                    type: DefaultPaginator
                    page_token_option:
                      type: RequestOption
                      inject_into: request_parameter
                      field_name: offset
                    page_size_option:
                      type: RequestOption
                      field_name: limit
                      inject_into: request_parameter
                    pagination_strategy:
                      type: OffsetIncrement
                      page_size: 200
                  partition_router:
                    - type: SubstreamPartitionRouter
                      parent_stream_configs:
                        - type: ParentStreamConfig
                          parent_key: id
                          partition_field: id
                          stream:
                            type: DeclarativeStream
                            name: Organization
                            primary_key:
                              - id
                            schema_loader:
                              type: InlineSchemaLoader
                              schema:
                                $schema: http://json-schema.org/schema#
                                properties:
                                  contact_name:
                                    type: string
                                  country:
                                    type: string
                                  created_at:
                                    type: string
                                  id:
                                    type: string
                                  locality:
                                    type: string
                                  name:
                                    type: string
                                  organization:
                                    properties:
                                      accepted_term_version:
                                        type: string
                                      address_line_1:
                                        type: string
                                      administrative_district_level_1:
                                        type: string
                                      business_type:
                                        type: string
                                      configuration_settings:
                                        properties:
                                          notifications_enabled:
                                            type: boolean
                                        type: object
                                      contact_email:
                                        type: string
                                      contact_name:
                                        type: string
                                      contact_phone:
                                        type: string
                                      country:
                                        type: string
                                      createdByCaller:
                                        type: boolean
                                      created_at:
                                        type: string
                                      id:
                                        type: string
                                      is_agency:
                                        type: boolean
                                      locality:
                                        type: string
                                      marketing_optin:
                                        type: boolean
                                      my_display_name:
                                        type: string
                                      my_invited_email:
                                        type: string
                                      my_member_id:
                                        type: string
                                      name:
                                        type: string
                                      postal_code:
                                        type: string
                                      roles:
                                        items:
                                          type: string
                                        type: array
                                      state:
                                        type: string
                                      type:
                                        type: string
                                      updated_at:
                                        type: string
                                    type: object
                                  roles:
                                    items:
                                      type: string
                                    type: array
                                  updated_at:
                                    type: string
                                type: object
                            retriever:
                              type: SimpleRetriever
                              requester:
                                type: HttpRequester
                                url_base: https://adsapi.snapchat.com/v1/
                                path: me/organizations
                                http_method: GET
                                request_parameters: {}
                                request_headers: {}
                                authenticator:
                                  type: OAuthAuthenticator
                                  client_id: '{{ config[''client_id''] }}'
                                  grant_type: refresh_token
                                  client_secret: '{{ config[''client_secret''] }}'
                                  refresh_token: '{{ config[''client_refresh_token''] }}'
                                  refresh_request_body: {}
                                  token_refresh_endpoint: >-
                                    https://accounts.snapchat.com/login/oauth2/access_token
                                request_body_json: {}
                              record_selector:
                                type: RecordSelector
                                extractor:
                                  type: DpathExtractor
                                  field_path:
                                    - organizations
                              paginator:
                                type: NoPagination
                            transformations:
                              - type: RemoveFields
                                field_pointers:
                                  - - sub_request_status
                              - type: AddFields
                                fields:
                                  - path:
                                      - id
                                    value: '{{ record.organization.id }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - updated_at
                                    value: '{{ record.organization.updated_at }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - created_at
                                    value: '{{ record.organization.created_at }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - country
                                    value: '{{ record.organization.country }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - name
                                    value: '{{ record.organization.name }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - contact_name
                                    value: '{{ record.organization.contact_name }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - locality
                                    value: '{{ record.organization.locality }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - roles
                                    value: '{{ record.organization.roles }}'
                            incremental_sync:
                              type: DatetimeBasedCursor
                              cursor_field: updated_at
                              lookback_window: P28D
                              cursor_datetime_formats:
                                - '%Y-%m-%dT%H:%M:%S.%fZ'
                              datetime_format: '%Y-%m-%dT%H:%M:%S.%fZ'
                              start_datetime:
                                type: MinMaxDatetime
                                datetime: '{{ config[''start_date''] }}'
                                datetime_format: '%Y-%m-%d'
                              end_datetime:
                                type: MinMaxDatetime
                                datetime: '{{ now_utc().strftime(''%Y-%m-%dT%H:%M:%SZ'') }}'
                                datetime_format: '%Y-%m-%dT%H:%M:%SZ'
                transformations:
                  - type: RemoveFields
                    field_pointers:
                      - - sub_request_status
                  - type: AddFields
                    fields:
                      - path:
                          - ad_account_id
                        value: '{{ record.adaccount.id }}'
                  - type: AddFields
                    fields:
                      - path:
                          - name
                        value: '{{ record.adaccount.name }}'
                  - type: AddFields
                    fields:
                      - path:
                          - type
                        value: '{{ record.adaccount.type }}'
                  - type: AddFields
                    fields:
                      - path:
                          - status
                        value: '{{ record.adaccount.status }}'
                  - type: AddFields
                    fields:
                      - path:
                          - updated_at
                        value: '{{ record.adaccount.updated_at }}'
                  - type: AddFields
                    fields:
                      - path:
                          - created_at
                        value: '{{ record.adaccount.created_at }}'
                  - type: AddFields
                    fields:
                      - path:
                          - organization_id
                        value: '{{ record.adaccount.organization_id }}'
                incremental_sync:
                  type: DatetimeBasedCursor
                  cursor_field: updated_at
                  cursor_datetime_formats:
                    - '%Y-%m-%dT%H:%M:%S.%fZ'
                  datetime_format: '%Y-%m-%dT%H:%M:%S.%fZ'
                  start_datetime:
                    type: MinMaxDatetime
                    datetime: '{{ config[''start_date''] }}'
                    datetime_format: '%Y-%m-%d'
                  end_datetime:
                    type: MinMaxDatetime
                    datetime: '{{ now_utc().strftime(''%Y-%m-%dT%H:%M:%SZ'') }}'
                    datetime_format: '%Y-%m-%dT%H:%M:%SZ'
    transformations:
      - type: RemoveFields
        field_pointers:
          - - sub_request_status
      - type: AddFields
        fields:
          - path:
              - id
            value: '{{ record.ad.id }}'
      - type: AddFields
        fields:
          - path:
              - updated_at
            value: '{{ record.ad.updated_at }}'
      - type: AddFields
        fields:
          - path:
              - created_at
            value: '{{ record.ad.created_at }}'
      - type: AddFields
        fields:
          - path:
              - name
            value: '{{ record.ad.name }}'
      - type: AddFields
        fields:
          - path:
              - ad_squad_id
            value: '{{ record.ad.ad_squad_id }}'
      - type: AddFields
        fields:
          - path:
              - creative_id
            value: '{{ record.ad.creative_id }}'
      - type: AddFields
        fields:
          - path:
              - status
            value: '{{ record.ad.status }}'
  - type: DeclarativeStream
    name: AdaccountsStatsDaily
    primary_key:
      - id
      - granularity
      - start_time
    schema_loader:
      type: InlineSchemaLoader
      schema:
        $schema: http://json-schema.org/schema#
        properties:
          granularity:
            type: string
          id:
            type: string
          start_time:
            type: string
          timeseries_stat:
            properties:
              breakdown_stats:
                properties:
                  campaign:
                    items:
                      properties:
                        end_time:
                          type: string
                        granularity:
                          type: string
                        id:
                          type: string
                        start_time:
                          type: string
                        timeseries:
                          items:
                            properties:
                              end_time:
                                type: string
                              start_time:
                                type: string
                              stats:
                                properties:
                                  spend:
                                    type: number
                                type: object
                            type: object
                          type: array
                        type:
                          type: string
                      type: object
                    type: array
                type: object
              end_time:
                type: string
              finalized_data_end_time:
                type: string
              id:
                type: string
              paging:
                properties:
                  next_link:
                    type: string
                type: object
              start_time:
                type: string
              swipe_up_attribution_window:
                type: string
              type:
                type: string
              view_attribution_window:
                type: string
            type: object
        type: object
    retriever:
      type: SimpleRetriever
      requester:
        type: HttpRequester
        url_base: https://adsapi.snapchat.com/v1/
        path: adaccounts/{{ stream_partition.id }}/stats
        http_method: GET
        request_parameters:
          fields: spend
          end_time: '{{ config[''end_date''] or today_utc() }}'
          breakdown: campaign
          start_time: '{{ config[''start_date''] }}'
          granularity: DAY
          action_report_time: '{{ config[''action_report_time''] }}'
          view_attribution_window: '{{ config[''view_attribution_window''] }}'
          swipe_up_attribution_window: '{{ config[''swipe_up_attribution_window''] }}'
        request_headers: {}
        authenticator:
          type: OAuthAuthenticator
          client_id: '{{ config[''client_id''] }}'
          grant_type: refresh_token
          client_secret: '{{ config[''client_secret''] }}'
          refresh_token: '{{ config[''client_refresh_token''] }}'
          refresh_request_body: {}
          token_refresh_endpoint: https://accounts.snapchat.com/login/oauth2/access_token
        request_body_json: {}
      record_selector:
        type: RecordSelector
        extractor:
          type: DpathExtractor
          field_path:
            - timeseries_stats
      paginator:
        type: DefaultPaginator
        page_size_option:
          type: RequestOption
          field_name: limit
          inject_into: request_parameter
        pagination_strategy:
          type: OffsetIncrement
          page_size: 200
      partition_router:
        - type: SubstreamPartitionRouter
          parent_stream_configs:
            - type: ParentStreamConfig
              parent_key: ad_account_id
              partition_field: id
              stream:
                type: DeclarativeStream
                name: Adaccounts
                primary_key:
                  - ad_account_id
                schema_loader:
                  type: InlineSchemaLoader
                  schema:
                    $schema: http://json-schema.org/schema#
                    properties:
                      ad_account_id:
                        type: string
                      adaccount:
                        properties:
                          advertiser_organization_id:
                            type: string
                          agency_client_metadata:
                            type: object
                          agency_representing_client:
                            type: boolean
                          billing_center_id:
                            type: string
                          billing_type:
                            type: string
                          cell_ids:
                            items:
                              type: string
                            type: array
                          client_paying_invoices:
                            type: boolean
                          created_at:
                            type: string
                          currency:
                            type: string
                          funding_source_ids:
                            items:
                              type: string
                            type: array
                          id:
                            type: string
                          name:
                            type: string
                          organization_id:
                            type: string
                          regulations:
                            properties:
                              restricted_delivery_signals:
                                type: boolean
                            type: object
                          status:
                            type: string
                          timezone:
                            type: string
                          type:
                            type: string
                          updated_at:
                            type: string
                        type: object
                      created_at:
                        type: string
                      name:
                        type: string
                      organization_id:
                        type: string
                      status:
                        type: string
                      type:
                        type: string
                      updated_at:
                        type: string
                    type: object
                retriever:
                  type: SimpleRetriever
                  requester:
                    type: HttpRequester
                    url_base: https://adsapi.snapchat.com/v1/
                    path: organizations/{{ stream_partition.id }}/adaccounts
                    http_method: GET
                    request_parameters: {}
                    request_headers: {}
                    authenticator:
                      type: OAuthAuthenticator
                      client_id: '{{ config[''client_id''] }}'
                      grant_type: refresh_token
                      client_secret: '{{ config[''client_secret''] }}'
                      refresh_token: '{{ config[''client_refresh_token''] }}'
                      refresh_request_body: {}
                      token_refresh_endpoint: https://accounts.snapchat.com/login/oauth2/access_token
                    error_handler:
                      type: CompositeErrorHandler
                      error_handlers:
                        - type: DefaultErrorHandler
                          backoff_strategies:
                            - type: ConstantBackoffStrategy
                              backoff_time_in_seconds: 5
                          response_filters:
                            - type: HttpResponseFilter
                              action: RETRY
                              http_codes:
                                - 429
                                - 500
                              error_message: Too Many Request
                    request_body_json: {}
                  record_selector:
                    type: RecordSelector
                    extractor:
                      type: DpathExtractor
                      field_path:
                        - adaccounts
                  paginator:
                    type: DefaultPaginator
                    page_token_option:
                      type: RequestOption
                      inject_into: request_parameter
                      field_name: offset
                    page_size_option:
                      type: RequestOption
                      field_name: limit
                      inject_into: request_parameter
                    pagination_strategy:
                      type: OffsetIncrement
                      page_size: 200
                  partition_router:
                    - type: SubstreamPartitionRouter
                      parent_stream_configs:
                        - type: ParentStreamConfig
                          parent_key: id
                          partition_field: id
                          stream:
                            type: DeclarativeStream
                            name: Organization
                            primary_key:
                              - id
                            schema_loader:
                              type: InlineSchemaLoader
                              schema:
                                $schema: http://json-schema.org/schema#
                                properties:
                                  contact_name:
                                    type: string
                                  country:
                                    type: string
                                  created_at:
                                    type: string
                                  id:
                                    type: string
                                  locality:
                                    type: string
                                  name:
                                    type: string
                                  organization:
                                    properties:
                                      accepted_term_version:
                                        type: string
                                      address_line_1:
                                        type: string
                                      administrative_district_level_1:
                                        type: string
                                      business_type:
                                        type: string
                                      configuration_settings:
                                        properties:
                                          notifications_enabled:
                                            type: boolean
                                        type: object
                                      contact_email:
                                        type: string
                                      contact_name:
                                        type: string
                                      contact_phone:
                                        type: string
                                      country:
                                        type: string
                                      createdByCaller:
                                        type: boolean
                                      created_at:
                                        type: string
                                      id:
                                        type: string
                                      is_agency:
                                        type: boolean
                                      locality:
                                        type: string
                                      marketing_optin:
                                        type: boolean
                                      my_display_name:
                                        type: string
                                      my_invited_email:
                                        type: string
                                      my_member_id:
                                        type: string
                                      name:
                                        type: string
                                      postal_code:
                                        type: string
                                      roles:
                                        items:
                                          type: string
                                        type: array
                                      state:
                                        type: string
                                      type:
                                        type: string
                                      updated_at:
                                        type: string
                                    type: object
                                  roles:
                                    items:
                                      type: string
                                    type: array
                                  updated_at:
                                    type: string
                                type: object
                            retriever:
                              type: SimpleRetriever
                              requester:
                                type: HttpRequester
                                url_base: https://adsapi.snapchat.com/v1/
                                path: me/organizations
                                http_method: GET
                                request_parameters: {}
                                request_headers: {}
                                authenticator:
                                  type: OAuthAuthenticator
                                  client_id: '{{ config[''client_id''] }}'
                                  grant_type: refresh_token
                                  client_secret: '{{ config[''client_secret''] }}'
                                  refresh_token: '{{ config[''client_refresh_token''] }}'
                                  refresh_request_body: {}
                                  token_refresh_endpoint: >-
                                    https://accounts.snapchat.com/login/oauth2/access_token
                                request_body_json: {}
                              record_selector:
                                type: RecordSelector
                                extractor:
                                  type: DpathExtractor
                                  field_path:
                                    - organizations
                              paginator:
                                type: NoPagination
                            transformations:
                              - type: RemoveFields
                                field_pointers:
                                  - - sub_request_status
                              - type: AddFields
                                fields:
                                  - path:
                                      - id
                                    value: '{{ record.organization.id }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - updated_at
                                    value: '{{ record.organization.updated_at }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - created_at
                                    value: '{{ record.organization.created_at }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - country
                                    value: '{{ record.organization.country }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - name
                                    value: '{{ record.organization.name }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - contact_name
                                    value: '{{ record.organization.contact_name }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - locality
                                    value: '{{ record.organization.locality }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - roles
                                    value: '{{ record.organization.roles }}'
                            incremental_sync:
                              type: DatetimeBasedCursor
                              cursor_field: updated_at
                              lookback_window: P28D
                              cursor_datetime_formats:
                                - '%Y-%m-%dT%H:%M:%S.%fZ'
                              datetime_format: '%Y-%m-%dT%H:%M:%S.%fZ'
                              start_datetime:
                                type: MinMaxDatetime
                                datetime: '{{ config[''start_date''] }}'
                                datetime_format: '%Y-%m-%d'
                              end_datetime:
                                type: MinMaxDatetime
                                datetime: '{{ now_utc().strftime(''%Y-%m-%dT%H:%M:%SZ'') }}'
                                datetime_format: '%Y-%m-%dT%H:%M:%SZ'
                transformations:
                  - type: RemoveFields
                    field_pointers:
                      - - sub_request_status
                  - type: AddFields
                    fields:
                      - path:
                          - ad_account_id
                        value: '{{ record.adaccount.id }}'
                  - type: AddFields
                    fields:
                      - path:
                          - name
                        value: '{{ record.adaccount.name }}'
                  - type: AddFields
                    fields:
                      - path:
                          - type
                        value: '{{ record.adaccount.type }}'
                  - type: AddFields
                    fields:
                      - path:
                          - status
                        value: '{{ record.adaccount.status }}'
                  - type: AddFields
                    fields:
                      - path:
                          - updated_at
                        value: '{{ record.adaccount.updated_at }}'
                  - type: AddFields
                    fields:
                      - path:
                          - created_at
                        value: '{{ record.adaccount.created_at }}'
                  - type: AddFields
                    fields:
                      - path:
                          - organization_id
                        value: '{{ record.adaccount.organization_id }}'
                incremental_sync:
                  type: DatetimeBasedCursor
                  cursor_field: updated_at
                  cursor_datetime_formats:
                    - '%Y-%m-%dT%H:%M:%S.%fZ'
                  datetime_format: '%Y-%m-%dT%H:%M:%S.%fZ'
                  start_datetime:
                    type: MinMaxDatetime
                    datetime: '{{ config[''start_date''] }}'
                    datetime_format: '%Y-%m-%d'
                  end_datetime:
                    type: MinMaxDatetime
                    datetime: '{{ now_utc().strftime(''%Y-%m-%dT%H:%M:%SZ'') }}'
                    datetime_format: '%Y-%m-%dT%H:%M:%SZ'
    transformations:
      - type: RemoveFields
        field_pointers:
          - - sub_request_status
      - type: AddFields
        fields:
          - path:
              - id
            value: '{{ record.timeseries_stat.id }}'
      - type: AddFields
        fields:
          - path:
              - granularity
            value: '{{ record.timeseries_stat.granularity }}'
      - type: AddFields
        fields:
          - path:
              - start_time
            value: '{{ record.timeseries_stat.start_time }}'
    incremental_sync:
      type: DatetimeBasedCursor
      cursor_field: start_time
      lookback_window: P28D
      cursor_datetime_formats:
        - '%Y-%m-%dT%H:%M:%S.%f%z'
      datetime_format: '%Y-%m-%d'
      start_datetime:
        type: MinMaxDatetime
        datetime: '{{ config[''start_date''] }}'
        datetime_format: '%Y-%m-%d'
      start_time_option:
        type: RequestOption
        field_name: start_time
        inject_into: request_parameter
      end_time_option:
        type: RequestOption
        field_name: end_time
        inject_into: request_parameter
      end_datetime:
        type: MinMaxDatetime
        datetime: '{{ config[''end_date''] or today_utc() }}'
        datetime_format: '%Y-%m-%d'
      step: P7D
      cursor_granularity: PT1S
  - type: DeclarativeStream
    name: AdsStatsDaily
    primary_key:
      - id
      - granularity
      - start_time
    schema_loader:
      type: InlineSchemaLoader
      schema:
        $schema: http://json-schema.org/schema#
        properties:
          granularity:
            type: string
          id:
            type: string
          start_time:
            type: string
          timeseries_stat:
            properties:
              conversion_data_processed_end_time:
                type: string
              end_time:
                type: string
              finalized_data_end_time:
                type: string
              granularity:
                type: string
              id:
                type: string
              start_time:
                type: string
              swipe_up_attribution_window:
                type: string
              timeseries:
                items:
                  properties:
                    end_time:
                      type: string
                    start_time:
                      type: string
                    stats:
                      properties:
                        android_installs:
                          type: number
                        attachment_avg_view_time_millis:
                          type: number
                        attachment_impressions:
                          type: number
                        attachment_quartile_1:
                          type: number
                        attachment_quartile_2:
                          type: number
                        attachment_quartile_3:
                          type: number
                        attachment_total_view_time_millis:
                          type: number
                        attachment_view_completion:
                          type: number
                        avg_screen_time_millis:
                          type: number
                        avg_view_time_millis:
                          type: number
                        conversion_achievement_unlocked:
                          type: number
                        conversion_ad_click:
                          type: number
                        conversion_ad_view:
                          type: number
                        conversion_add_billing:
                          type: number
                        conversion_add_cart:
                          type: number
                        conversion_add_to_wishlist:
                          type: number
                        conversion_app_opens:
                          type: number
                        conversion_complete_tutorial:
                          type: number
                        conversion_invite:
                          type: number
                        conversion_level_completes:
                          type: number
                        conversion_list_view:
                          type: number
                        conversion_login:
                          type: number
                        conversion_page_views:
                          type: number
                        conversion_purchases:
                          type: number
                        conversion_purchases_value:
                          type: number
                        conversion_rate:
                          type: number
                        conversion_reserve:
                          type: number
                        conversion_save:
                          type: number
                        conversion_searches:
                          type: number
                        conversion_share:
                          type: number
                        conversion_spend_credits:
                          type: number
                        conversion_start_checkout:
                          type: number
                        conversion_start_trial:
                          type: number
                        conversion_subscribe:
                          type: number
                        conversion_view_content:
                          type: number
                        custom_event_1:
                          type: number
                        custom_event_2:
                          type: number
                        custom_event_3:
                          type: number
                        custom_event_4:
                          type: number
                        custom_event_5:
                          type: number
                        earned_impressions:
                          type: number
                        impressions:
                          type: number
                        ios_installs:
                          type: number
                        native_leads:
                          type: number
                        paid_impressions:
                          type: number
                        play_time_millis:
                          type: number
                        quartile_1:
                          type: number
                        quartile_2:
                          type: number
                        quartile_3:
                          type: number
                        saves:
                          type: number
                        screen_time_millis:
                          type: number
                        shares:
                          type: number
                        spend:
                          type: number
                        story_completes:
                          type: number
                        story_opens:
                          type: number
                        swipe_up_percent:
                          type: number
                        swipes:
                          type: number
                        total_impressions:
                          type: number
                        total_installs:
                          type: number
                        video_views:
                          type: number
                        video_views_15s:
                          type: number
                        video_views_time_based:
                          type: number
                        view_completion:
                          type: number
                        view_time_millis:
                          type: number
                      type: object
                  type: object
                type: array
              type:
                type: string
              view_attribution_window:
                type: string
            type: object
        type: object
    retriever:
      type: SimpleRetriever
      requester:
        type: HttpRequester
        url_base: https://adsapi.snapchat.com/v1/
        path: ads/{{ stream_interval.id }}/stats
        http_method: GET
        request_parameters:
          fields: >-
            android_installs,attachment_avg_view_time_millis,attachment_impressions,attachment_quartile_1,attachment_quartile_2,attachment_quartile_3,attachment_total_view_time_millis,attachment_view_completion,avg_screen_time_millis,avg_view_time_millis,impressions,ios_installs,quartile_1,quartile_2,quartile_3,screen_time_millis,spend,swipe_up_percent,swipes,total_installs,video_views,video_views_time_based,video_views_15s,view_completion,view_time_millis,paid_impressions,earned_impressions,total_impressions,play_time_millis,shares,saves,native_leads,conversion_purchases,conversion_purchases_value,conversion_save,conversion_start_checkout,conversion_add_cart,conversion_view_content,conversion_add_billing,conversion_searches,conversion_level_completes,conversion_app_opens,conversion_page_views,conversion_subscribe,conversion_ad_click,conversion_ad_view,conversion_complete_tutorial,conversion_invite,conversion_login,conversion_share,conversion_reserve,conversion_achievement_unlocked,conversion_add_to_wishlist,conversion_spend_credits,conversion_rate,conversion_start_trial,conversion_list_view,custom_event_1,custom_event_2,custom_event_3,custom_event_4,custom_event_5,story_opens,story_completes
          end_time: '{{ config[''end_date''] or today_utc() }}'
          start_time: '{{ config[''start_date''] }}'
          granularity: DAY
          action_report_time: '{{ config[''action_report_time''] }}'
          view_attribution_window: '{{ config[''view_attribution_window''] }}'
          swipe_up_attribution_window: '{{ config[''swipe_up_attribution_window''] }}'
        request_headers: {}
        authenticator:
          type: OAuthAuthenticator
          client_id: '{{ config[''client_id''] }}'
          grant_type: refresh_token
          client_secret: '{{ config[''client_secret''] }}'
          refresh_token: '{{ config[''client_refresh_token''] }}'
          refresh_request_body: {}
          token_refresh_endpoint: https://accounts.snapchat.com/login/oauth2/access_token
        error_handler:
          type: CompositeErrorHandler
          error_handlers:
            - type: DefaultErrorHandler
              backoff_strategies:
                - type: ConstantBackoffStrategy
                  backoff_time_in_seconds: 5
        request_body_json: {}
      record_selector:
        type: RecordSelector
        extractor:
          type: DpathExtractor
          field_path:
            - timeseries_stats
      paginator:
        type: DefaultPaginator
        page_size_option:
          type: RequestOption
          field_name: limit
          inject_into: request_parameter
        pagination_strategy:
          type: OffsetIncrement
          page_size: 200
      partition_router:
        - type: SubstreamPartitionRouter
          parent_stream_configs:
            - type: ParentStreamConfig
              parent_key: id
              partition_field: id
              stream:
                type: DeclarativeStream
                name: Ads
                primary_key:
                  - id
                schema_loader:
                  type: InlineSchemaLoader
                  schema:
                    $schema: http://json-schema.org/schema#
                    properties:
                      ad:
                        properties:
                          ad_squad_id:
                            type: string
                          created_at:
                            type: string
                          creative_id:
                            type: string
                          delivery_status:
                            items:
                              type: string
                            type: array
                          id:
                            type: string
                          name:
                            type: string
                          render_type:
                            type: string
                          review_status:
                            type: string
                          status:
                            type: string
                          type:
                            type: string
                          updated_at:
                            type: string
                        type: object
                      ad_squad_id:
                        type: string
                      created_at:
                        type: string
                      creative_id:
                        type: string
                      id:
                        type: string
                      name:
                        type: string
                      status:
                        type: string
                      updated_at:
                        type: string
                    type: object
                retriever:
                  type: SimpleRetriever
                  requester:
                    type: HttpRequester
                    url_base: https://adsapi.snapchat.com/v1/
                    path: adaccounts/{{ stream_partition.id }}/ads
                    http_method: GET
                    request_parameters: {}
                    request_headers: {}
                    authenticator:
                      type: OAuthAuthenticator
                      client_id: '{{ config[''client_id''] }}'
                      grant_type: refresh_token
                      client_secret: '{{ config[''client_secret''] }}'
                      refresh_token: '{{ config[''client_refresh_token''] }}'
                      refresh_request_body: {}
                      token_refresh_endpoint: https://accounts.snapchat.com/login/oauth2/access_token
                    request_body_json: {}
                  record_selector:
                    type: RecordSelector
                    extractor:
                      type: DpathExtractor
                      field_path:
                        - ads
                  paginator:
                    type: DefaultPaginator
                    page_size_option:
                      type: RequestOption
                      field_name: limit
                      inject_into: request_parameter
                    pagination_strategy:
                      type: OffsetIncrement
                      page_size: 200
                  partition_router:
                    - type: SubstreamPartitionRouter
                      parent_stream_configs:
                        - type: ParentStreamConfig
                          parent_key: ad_account_id
                          partition_field: id
                          stream:
                            type: DeclarativeStream
                            name: Adaccounts
                            primary_key:
                              - ad_account_id
                            schema_loader:
                              type: InlineSchemaLoader
                              schema:
                                $schema: http://json-schema.org/schema#
                                properties:
                                  ad_account_id:
                                    type: string
                                  adaccount:
                                    properties:
                                      advertiser_organization_id:
                                        type: string
                                      agency_client_metadata:
                                        type: object
                                      agency_representing_client:
                                        type: boolean
                                      billing_center_id:
                                        type: string
                                      billing_type:
                                        type: string
                                      cell_ids:
                                        items:
                                          type: string
                                        type: array
                                      client_paying_invoices:
                                        type: boolean
                                      created_at:
                                        type: string
                                      currency:
                                        type: string
                                      funding_source_ids:
                                        items:
                                          type: string
                                        type: array
                                      id:
                                        type: string
                                      name:
                                        type: string
                                      organization_id:
                                        type: string
                                      regulations:
                                        properties:
                                          restricted_delivery_signals:
                                            type: boolean
                                        type: object
                                      status:
                                        type: string
                                      timezone:
                                        type: string
                                      type:
                                        type: string
                                      updated_at:
                                        type: string
                                    type: object
                                  created_at:
                                    type: string
                                  name:
                                    type: string
                                  organization_id:
                                    type: string
                                  status:
                                    type: string
                                  type:
                                    type: string
                                  updated_at:
                                    type: string
                                type: object
                            retriever:
                              type: SimpleRetriever
                              requester:
                                type: HttpRequester
                                url_base: https://adsapi.snapchat.com/v1/
                                path: >-
                                  organizations/{{ stream_partition.id
                                  }}/adaccounts
                                http_method: GET
                                request_parameters: {}
                                request_headers: {}
                                authenticator:
                                  type: OAuthAuthenticator
                                  client_id: '{{ config[''client_id''] }}'
                                  grant_type: refresh_token
                                  client_secret: '{{ config[''client_secret''] }}'
                                  refresh_token: '{{ config[''client_refresh_token''] }}'
                                  refresh_request_body: {}
                                  token_refresh_endpoint: >-
                                    https://accounts.snapchat.com/login/oauth2/access_token
                                error_handler:
                                  type: CompositeErrorHandler
                                  error_handlers:
                                    - type: DefaultErrorHandler
                                      backoff_strategies:
                                        - type: ConstantBackoffStrategy
                                          backoff_time_in_seconds: 5
                                      response_filters:
                                        - type: HttpResponseFilter
                                          action: RETRY
                                          http_codes:
                                            - 429
                                            - 500
                                          error_message: Too Many Request
                                request_body_json: {}
                              record_selector:
                                type: RecordSelector
                                extractor:
                                  type: DpathExtractor
                                  field_path:
                                    - adaccounts
                              paginator:
                                type: DefaultPaginator
                                page_token_option:
                                  type: RequestOption
                                  inject_into: request_parameter
                                  field_name: offset
                                page_size_option:
                                  type: RequestOption
                                  field_name: limit
                                  inject_into: request_parameter
                                pagination_strategy:
                                  type: OffsetIncrement
                                  page_size: 200
                              partition_router:
                                - type: SubstreamPartitionRouter
                                  parent_stream_configs:
                                    - type: ParentStreamConfig
                                      parent_key: id
                                      partition_field: id
                                      stream:
                                        type: DeclarativeStream
                                        name: Organization
                                        primary_key:
                                          - id
                                        schema_loader:
                                          type: InlineSchemaLoader
                                          schema:
                                            $schema: http://json-schema.org/schema#
                                            properties:
                                              contact_name:
                                                type: string
                                              country:
                                                type: string
                                              created_at:
                                                type: string
                                              id:
                                                type: string
                                              locality:
                                                type: string
                                              name:
                                                type: string
                                              organization:
                                                properties:
                                                  accepted_term_version:
                                                    type: string
                                                  address_line_1:
                                                    type: string
                                                  administrative_district_level_1:
                                                    type: string
                                                  business_type:
                                                    type: string
                                                  configuration_settings:
                                                    properties:
                                                      notifications_enabled:
                                                        type: boolean
                                                    type: object
                                                  contact_email:
                                                    type: string
                                                  contact_name:
                                                    type: string
                                                  contact_phone:
                                                    type: string
                                                  country:
                                                    type: string
                                                  createdByCaller:
                                                    type: boolean
                                                  created_at:
                                                    type: string
                                                  id:
                                                    type: string
                                                  is_agency:
                                                    type: boolean
                                                  locality:
                                                    type: string
                                                  marketing_optin:
                                                    type: boolean
                                                  my_display_name:
                                                    type: string
                                                  my_invited_email:
                                                    type: string
                                                  my_member_id:
                                                    type: string
                                                  name:
                                                    type: string
                                                  postal_code:
                                                    type: string
                                                  roles:
                                                    items:
                                                      type: string
                                                    type: array
                                                  state:
                                                    type: string
                                                  type:
                                                    type: string
                                                  updated_at:
                                                    type: string
                                                type: object
                                              roles:
                                                items:
                                                  type: string
                                                type: array
                                              updated_at:
                                                type: string
                                            type: object
                                        retriever:
                                          type: SimpleRetriever
                                          requester:
                                            type: HttpRequester
                                            url_base: https://adsapi.snapchat.com/v1/
                                            path: me/organizations
                                            http_method: GET
                                            request_parameters: {}
                                            request_headers: {}
                                            authenticator:
                                              type: OAuthAuthenticator
                                              client_id: '{{ config[''client_id''] }}'
                                              grant_type: refresh_token
                                              client_secret: '{{ config[''client_secret''] }}'
                                              refresh_token: '{{ config[''client_refresh_token''] }}'
                                              refresh_request_body: {}
                                              token_refresh_endpoint: >-
                                                https://accounts.snapchat.com/login/oauth2/access_token
                                            request_body_json: {}
                                          record_selector:
                                            type: RecordSelector
                                            extractor:
                                              type: DpathExtractor
                                              field_path:
                                                - organizations
                                          paginator:
                                            type: NoPagination
                                        transformations:
                                          - type: RemoveFields
                                            field_pointers:
                                              - - sub_request_status
                                          - type: AddFields
                                            fields:
                                              - path:
                                                  - id
                                                value: '{{ record.organization.id }}'
                                          - type: AddFields
                                            fields:
                                              - path:
                                                  - updated_at
                                                value: '{{ record.organization.updated_at }}'
                                          - type: AddFields
                                            fields:
                                              - path:
                                                  - created_at
                                                value: '{{ record.organization.created_at }}'
                                          - type: AddFields
                                            fields:
                                              - path:
                                                  - country
                                                value: '{{ record.organization.country }}'
                                          - type: AddFields
                                            fields:
                                              - path:
                                                  - name
                                                value: '{{ record.organization.name }}'
                                          - type: AddFields
                                            fields:
                                              - path:
                                                  - contact_name
                                                value: '{{ record.organization.contact_name }}'
                                          - type: AddFields
                                            fields:
                                              - path:
                                                  - locality
                                                value: '{{ record.organization.locality }}'
                                          - type: AddFields
                                            fields:
                                              - path:
                                                  - roles
                                                value: '{{ record.organization.roles }}'
                                        incremental_sync:
                                          type: DatetimeBasedCursor
                                          cursor_field: updated_at
                                          lookback_window: P28D
                                          cursor_datetime_formats:
                                            - '%Y-%m-%dT%H:%M:%S.%fZ'
                                          datetime_format: '%Y-%m-%dT%H:%M:%S.%fZ'
                                          start_datetime:
                                            type: MinMaxDatetime
                                            datetime: '{{ config[''start_date''] }}'
                                            datetime_format: '%Y-%m-%d'
                                          end_datetime:
                                            type: MinMaxDatetime
                                            datetime: >-
                                              {{
                                              now_utc().strftime('%Y-%m-%dT%H:%M:%SZ')
                                              }}
                                            datetime_format: '%Y-%m-%dT%H:%M:%SZ'
                            transformations:
                              - type: RemoveFields
                                field_pointers:
                                  - - sub_request_status
                              - type: AddFields
                                fields:
                                  - path:
                                      - ad_account_id
                                    value: '{{ record.adaccount.id }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - name
                                    value: '{{ record.adaccount.name }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - type
                                    value: '{{ record.adaccount.type }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - status
                                    value: '{{ record.adaccount.status }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - updated_at
                                    value: '{{ record.adaccount.updated_at }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - created_at
                                    value: '{{ record.adaccount.created_at }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - organization_id
                                    value: '{{ record.adaccount.organization_id }}'
                            incremental_sync:
                              type: DatetimeBasedCursor
                              cursor_field: updated_at
                              cursor_datetime_formats:
                                - '%Y-%m-%dT%H:%M:%S.%fZ'
                              datetime_format: '%Y-%m-%dT%H:%M:%S.%fZ'
                              start_datetime:
                                type: MinMaxDatetime
                                datetime: '{{ config[''start_date''] }}'
                                datetime_format: '%Y-%m-%d'
                              end_datetime:
                                type: MinMaxDatetime
                                datetime: '{{ now_utc().strftime(''%Y-%m-%dT%H:%M:%SZ'') }}'
                                datetime_format: '%Y-%m-%dT%H:%M:%SZ'
                transformations:
                  - type: RemoveFields
                    field_pointers:
                      - - sub_request_status
                  - type: AddFields
                    fields:
                      - path:
                          - id
                        value: '{{ record.ad.id }}'
                  - type: AddFields
                    fields:
                      - path:
                          - updated_at
                        value: '{{ record.ad.updated_at }}'
                  - type: AddFields
                    fields:
                      - path:
                          - created_at
                        value: '{{ record.ad.created_at }}'
                  - type: AddFields
                    fields:
                      - path:
                          - name
                        value: '{{ record.ad.name }}'
                  - type: AddFields
                    fields:
                      - path:
                          - ad_squad_id
                        value: '{{ record.ad.ad_squad_id }}'
                  - type: AddFields
                    fields:
                      - path:
                          - creative_id
                        value: '{{ record.ad.creative_id }}'
                  - type: AddFields
                    fields:
                      - path:
                          - status
                        value: '{{ record.ad.status }}'
    transformations:
      - type: RemoveFields
        field_pointers:
          - - sub_request_status
      - type: AddFields
        fields:
          - path:
              - id
            value: '{{ record.timeseries_stat.id }}'
      - type: AddFields
        fields:
          - path:
              - start_time
            value: '{{ record.timeseries_stat.start_time }}'
      - type: AddFields
        fields:
          - path:
              - granularity
            value: '{{ record.timeseries_stat.granularity }}'
    incremental_sync:
      type: DatetimeBasedCursor
      cursor_field: start_time
      lookback_window: P28D
      cursor_datetime_formats:
        - '%Y-%m-%d'
      datetime_format: '%Y-%m-%d'
      start_datetime:
        type: MinMaxDatetime
        datetime: '{{ config[''start_date''] }}'
        datetime_format: '%Y-%m-%dT%H:%M:%SZ'
      start_time_option:
        type: RequestOption
        field_name: start_time
        inject_into: request_parameter
      end_time_option:
        type: RequestOption
        field_name: end_time
        inject_into: request_parameter
      end_datetime:
        type: MinMaxDatetime
        datetime: '{{ now_utc().strftime(''%Y-%m-%dT%H:%M:%SZ'') }}'
        datetime_format: '%Y-%m-%dT%H:%M:%SZ'
      step: P31D
      cursor_granularity: P1D
  - type: DeclarativeStream
    name: CampaignsStatsDaily
    primary_key:
      - id
      - granularity
      - start_time
    schema_loader:
      type: InlineSchemaLoader
      schema:
        $schema: http://json-schema.org/schema#
        properties:
          granularity:
            type: string
          id:
            type: string
          start_time:
            type: string
          timeseries_stat:
            properties:
              breakdown_stats:
                properties:
                  ad:
                    type: array
                type: object
              end_time:
                type: string
              finalized_data_end_time:
                type: string
              id:
                type: string
              paging:
                properties:
                  next_link:
                    type: string
                type: object
              start_time:
                type: string
              swipe_up_attribution_window:
                type: string
              type:
                type: string
              view_attribution_window:
                type: string
            type: object
        type: object
    retriever:
      type: SimpleRetriever
      requester:
        type: HttpRequester
        url_base: https://adsapi.snapchat.com/v1/
        path: campaigns/{{ stream_partition.id }}/stats
        http_method: GET
        request_parameters:
          fields: >-
            android_installs,attachment_avg_view_time_millis,attachment_impressions,attachment_quartile_1,attachment_quartile_2,attachment_quartile_3,attachment_total_view_time_millis,attachment_view_completion,avg_screen_time_millis,avg_view_time_millis,impressions,ios_installs,quartile_1,quartile_2,quartile_3,screen_time_millis,spend,swipe_up_percent,swipes,total_installs,video_views,video_views_time_based,video_views_15s,view_completion,view_time_millis,paid_impressions,earned_impressions,total_impressions,play_time_millis,shares,saves,native_leads,conversion_purchases,conversion_purchases_value,conversion_save,conversion_start_checkout,conversion_add_cart,conversion_view_content,conversion_add_billing,conversion_searches,conversion_level_completes,conversion_app_opens,conversion_page_views,conversion_subscribe,conversion_ad_click,conversion_ad_view,conversion_complete_tutorial,conversion_invite,conversion_login,conversion_share,conversion_reserve,conversion_achievement_unlocked,conversion_add_to_wishlist,conversion_spend_credits,conversion_rate,conversion_start_trial,conversion_list_view,custom_event_1,custom_event_2,custom_event_3,custom_event_4,custom_event_5,story_opens,story_completes
          end_time: '{{ config[''end_date''] or today_utc() }}'
          breakdown: AD
          start_time: '{{ config[''start_date''] }}'
          granularity: DAY
          action_report_time: '{{ config[''action_report_time''] }}'
          view_attribution_window: '{{ config[''view_attribution_window''] }}'
          swipe_up_attribution_window: '{{ config[''swipe_up_attribution_window''] }}'
        request_headers: {}
        authenticator:
          type: OAuthAuthenticator
          client_id: '{{ config[''client_id''] }}'
          grant_type: refresh_token
          client_secret: '{{ config[''client_secret''] }}'
          refresh_token: '{{ config[''client_refresh_token''] }}'
          refresh_request_body: {}
          token_refresh_endpoint: https://accounts.snapchat.com/login/oauth2/access_token
        error_handler:
          type: CompositeErrorHandler
          error_handlers:
            - type: DefaultErrorHandler
              backoff_strategies:
                - type: ConstantBackoffStrategy
                  backoff_time_in_seconds: 5
        request_body_json: {}
      record_selector:
        type: RecordSelector
        extractor:
          type: DpathExtractor
          field_path:
            - timeseries_stats
      paginator:
        type: DefaultPaginator
        page_size_option:
          type: RequestOption
          field_name: limit
          inject_into: request_parameter
        pagination_strategy:
          type: OffsetIncrement
          page_size: 200
      partition_router:
        - type: SubstreamPartitionRouter
          parent_stream_configs:
            - type: ParentStreamConfig
              parent_key: id
              partition_field: id
              stream:
                type: DeclarativeStream
                name: Campaigns
                primary_key:
                  - id
                schema_loader:
                  type: InlineSchemaLoader
                  schema:
                    $schema: http://json-schema.org/schema#
                    properties:
                      ad_account_id:
                        type: string
                      campaign:
                        properties:
                          ad_account_id:
                            type: string
                          buy_model:
                            type: string
                          created_at:
                            type: string
                          creation_state:
                            type: string
                          delivery_status:
                            items:
                              type: string
                            type: array
                          end_time:
                            type: string
                          id:
                            type: string
                          lifetime_spend_cap_micro:
                            type: number
                          name:
                            type: string
                          objective:
                            type: string
                          start_time:
                            type: string
                          status:
                            type: string
                          updated_at:
                            type: string
                        type: object
                      created_at:
                        type: string
                      id:
                        type: string
                      name:
                        type: string
                      objective:
                        type: string
                      start_time:
                        type: string
                      status:
                        type: string
                      updated_at:
                        type: string
                    type: object
                retriever:
                  type: SimpleRetriever
                  requester:
                    type: HttpRequester
                    url_base: https://adsapi.snapchat.com/v1/
                    path: adaccounts/{{ stream_partition.id }}/campaigns
                    http_method: GET
                    request_parameters: {}
                    request_headers: {}
                    authenticator:
                      type: OAuthAuthenticator
                      client_id: '{{ config[''client_id''] }}'
                      grant_type: refresh_token
                      client_secret: '{{ config[''client_secret''] }}'
                      refresh_token: '{{ config[''client_refresh_token''] }}'
                      refresh_request_body: {}
                      token_refresh_endpoint: https://accounts.snapchat.com/login/oauth2/access_token
                    request_body_json: {}
                  record_selector:
                    type: RecordSelector
                    extractor:
                      type: DpathExtractor
                      field_path:
                        - campaigns
                  paginator:
                    type: DefaultPaginator
                    page_token_option:
                      type: RequestOption
                      inject_into: request_parameter
                      field_name: offset
                    page_size_option:
                      type: RequestOption
                      field_name: limit
                      inject_into: request_parameter
                    pagination_strategy:
                      type: OffsetIncrement
                      page_size: 200
                  partition_router:
                    - type: SubstreamPartitionRouter
                      parent_stream_configs:
                        - type: ParentStreamConfig
                          parent_key: ad_account_id
                          partition_field: id
                          stream:
                            type: DeclarativeStream
                            name: Adaccounts
                            primary_key:
                              - ad_account_id
                            schema_loader:
                              type: InlineSchemaLoader
                              schema:
                                $schema: http://json-schema.org/schema#
                                properties:
                                  ad_account_id:
                                    type: string
                                  adaccount:
                                    properties:
                                      advertiser_organization_id:
                                        type: string
                                      agency_client_metadata:
                                        type: object
                                      agency_representing_client:
                                        type: boolean
                                      billing_center_id:
                                        type: string
                                      billing_type:
                                        type: string
                                      cell_ids:
                                        items:
                                          type: string
                                        type: array
                                      client_paying_invoices:
                                        type: boolean
                                      created_at:
                                        type: string
                                      currency:
                                        type: string
                                      funding_source_ids:
                                        items:
                                          type: string
                                        type: array
                                      id:
                                        type: string
                                      name:
                                        type: string
                                      organization_id:
                                        type: string
                                      regulations:
                                        properties:
                                          restricted_delivery_signals:
                                            type: boolean
                                        type: object
                                      status:
                                        type: string
                                      timezone:
                                        type: string
                                      type:
                                        type: string
                                      updated_at:
                                        type: string
                                    type: object
                                  created_at:
                                    type: string
                                  name:
                                    type: string
                                  organization_id:
                                    type: string
                                  status:
                                    type: string
                                  type:
                                    type: string
                                  updated_at:
                                    type: string
                                type: object
                            retriever:
                              type: SimpleRetriever
                              requester:
                                type: HttpRequester
                                url_base: https://adsapi.snapchat.com/v1/
                                path: >-
                                  organizations/{{ stream_partition.id
                                  }}/adaccounts
                                http_method: GET
                                request_parameters: {}
                                request_headers: {}
                                authenticator:
                                  type: OAuthAuthenticator
                                  client_id: '{{ config[''client_id''] }}'
                                  grant_type: refresh_token
                                  client_secret: '{{ config[''client_secret''] }}'
                                  refresh_token: '{{ config[''client_refresh_token''] }}'
                                  refresh_request_body: {}
                                  token_refresh_endpoint: >-
                                    https://accounts.snapchat.com/login/oauth2/access_token
                                error_handler:
                                  type: CompositeErrorHandler
                                  error_handlers:
                                    - type: DefaultErrorHandler
                                      backoff_strategies:
                                        - type: ConstantBackoffStrategy
                                          backoff_time_in_seconds: 5
                                      response_filters:
                                        - type: HttpResponseFilter
                                          action: RETRY
                                          http_codes:
                                            - 429
                                            - 500
                                          error_message: Too Many Request
                                request_body_json: {}
                              record_selector:
                                type: RecordSelector
                                extractor:
                                  type: DpathExtractor
                                  field_path:
                                    - adaccounts
                              paginator:
                                type: DefaultPaginator
                                page_token_option:
                                  type: RequestOption
                                  inject_into: request_parameter
                                  field_name: offset
                                page_size_option:
                                  type: RequestOption
                                  field_name: limit
                                  inject_into: request_parameter
                                pagination_strategy:
                                  type: OffsetIncrement
                                  page_size: 200
                              partition_router:
                                - type: SubstreamPartitionRouter
                                  parent_stream_configs:
                                    - type: ParentStreamConfig
                                      parent_key: id
                                      partition_field: id
                                      stream:
                                        type: DeclarativeStream
                                        name: Organization
                                        primary_key:
                                          - id
                                        schema_loader:
                                          type: InlineSchemaLoader
                                          schema:
                                            $schema: http://json-schema.org/schema#
                                            properties:
                                              contact_name:
                                                type: string
                                              country:
                                                type: string
                                              created_at:
                                                type: string
                                              id:
                                                type: string
                                              locality:
                                                type: string
                                              name:
                                                type: string
                                              organization:
                                                properties:
                                                  accepted_term_version:
                                                    type: string
                                                  address_line_1:
                                                    type: string
                                                  administrative_district_level_1:
                                                    type: string
                                                  business_type:
                                                    type: string
                                                  configuration_settings:
                                                    properties:
                                                      notifications_enabled:
                                                        type: boolean
                                                    type: object
                                                  contact_email:
                                                    type: string
                                                  contact_name:
                                                    type: string
                                                  contact_phone:
                                                    type: string
                                                  country:
                                                    type: string
                                                  createdByCaller:
                                                    type: boolean
                                                  created_at:
                                                    type: string
                                                  id:
                                                    type: string
                                                  is_agency:
                                                    type: boolean
                                                  locality:
                                                    type: string
                                                  marketing_optin:
                                                    type: boolean
                                                  my_display_name:
                                                    type: string
                                                  my_invited_email:
                                                    type: string
                                                  my_member_id:
                                                    type: string
                                                  name:
                                                    type: string
                                                  postal_code:
                                                    type: string
                                                  roles:
                                                    items:
                                                      type: string
                                                    type: array
                                                  state:
                                                    type: string
                                                  type:
                                                    type: string
                                                  updated_at:
                                                    type: string
                                                type: object
                                              roles:
                                                items:
                                                  type: string
                                                type: array
                                              updated_at:
                                                type: string
                                            type: object
                                        retriever:
                                          type: SimpleRetriever
                                          requester:
                                            type: HttpRequester
                                            url_base: https://adsapi.snapchat.com/v1/
                                            path: me/organizations
                                            http_method: GET
                                            request_parameters: {}
                                            request_headers: {}
                                            authenticator:
                                              type: OAuthAuthenticator
                                              client_id: '{{ config[''client_id''] }}'
                                              grant_type: refresh_token
                                              client_secret: '{{ config[''client_secret''] }}'
                                              refresh_token: '{{ config[''client_refresh_token''] }}'
                                              refresh_request_body: {}
                                              token_refresh_endpoint: >-
                                                https://accounts.snapchat.com/login/oauth2/access_token
                                            request_body_json: {}
                                          record_selector:
                                            type: RecordSelector
                                            extractor:
                                              type: DpathExtractor
                                              field_path:
                                                - organizations
                                          paginator:
                                            type: NoPagination
                                        transformations:
                                          - type: RemoveFields
                                            field_pointers:
                                              - - sub_request_status
                                          - type: AddFields
                                            fields:
                                              - path:
                                                  - id
                                                value: '{{ record.organization.id }}'
                                          - type: AddFields
                                            fields:
                                              - path:
                                                  - updated_at
                                                value: '{{ record.organization.updated_at }}'
                                          - type: AddFields
                                            fields:
                                              - path:
                                                  - created_at
                                                value: '{{ record.organization.created_at }}'
                                          - type: AddFields
                                            fields:
                                              - path:
                                                  - country
                                                value: '{{ record.organization.country }}'
                                          - type: AddFields
                                            fields:
                                              - path:
                                                  - name
                                                value: '{{ record.organization.name }}'
                                          - type: AddFields
                                            fields:
                                              - path:
                                                  - contact_name
                                                value: '{{ record.organization.contact_name }}'
                                          - type: AddFields
                                            fields:
                                              - path:
                                                  - locality
                                                value: '{{ record.organization.locality }}'
                                          - type: AddFields
                                            fields:
                                              - path:
                                                  - roles
                                                value: '{{ record.organization.roles }}'
                                        incremental_sync:
                                          type: DatetimeBasedCursor
                                          cursor_field: updated_at
                                          lookback_window: P28D
                                          cursor_datetime_formats:
                                            - '%Y-%m-%dT%H:%M:%S.%fZ'
                                          datetime_format: '%Y-%m-%dT%H:%M:%S.%fZ'
                                          start_datetime:
                                            type: MinMaxDatetime
                                            datetime: '{{ config[''start_date''] }}'
                                            datetime_format: '%Y-%m-%d'
                                          end_datetime:
                                            type: MinMaxDatetime
                                            datetime: >-
                                              {{
                                              now_utc().strftime('%Y-%m-%dT%H:%M:%SZ')
                                              }}
                                            datetime_format: '%Y-%m-%dT%H:%M:%SZ'
                            transformations:
                              - type: RemoveFields
                                field_pointers:
                                  - - sub_request_status
                              - type: AddFields
                                fields:
                                  - path:
                                      - ad_account_id
                                    value: '{{ record.adaccount.id }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - name
                                    value: '{{ record.adaccount.name }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - type
                                    value: '{{ record.adaccount.type }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - status
                                    value: '{{ record.adaccount.status }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - updated_at
                                    value: '{{ record.adaccount.updated_at }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - created_at
                                    value: '{{ record.adaccount.created_at }}'
                              - type: AddFields
                                fields:
                                  - path:
                                      - organization_id
                                    value: '{{ record.adaccount.organization_id }}'
                            incremental_sync:
                              type: DatetimeBasedCursor
                              cursor_field: updated_at
                              cursor_datetime_formats:
                                - '%Y-%m-%dT%H:%M:%S.%fZ'
                              datetime_format: '%Y-%m-%dT%H:%M:%S.%fZ'
                              start_datetime:
                                type: MinMaxDatetime
                                datetime: '{{ config[''start_date''] }}'
                                datetime_format: '%Y-%m-%d'
                              end_datetime:
                                type: MinMaxDatetime
                                datetime: '{{ now_utc().strftime(''%Y-%m-%dT%H:%M:%SZ'') }}'
                                datetime_format: '%Y-%m-%dT%H:%M:%SZ'
                transformations:
                  - type: RemoveFields
                    field_pointers:
                      - - sub_request_status
                  - type: AddFields
                    fields:
                      - path:
                          - id
                        value: '{{ record.campaign.id }}'
                  - type: AddFields
                    fields:
                      - path:
                          - updated_at
                        value: '{{ record.campaign.updated_at }}'
                  - type: AddFields
                    fields:
                      - path:
                          - created_at
                        value: '{{ record.campaign.created_at }}'
                  - type: AddFields
                    fields:
                      - path:
                          - name
                        value: '{{ record.campaign.name }}'
                  - type: AddFields
                    fields:
                      - path:
                          - ad_account_id
                        value: '{{ record.campaign.ad_account_id }}'
                  - type: AddFields
                    fields:
                      - path:
                          - status
                        value: '{{ record.campaign.status }}'
                  - type: AddFields
                    fields:
                      - path:
                          - objective
                        value: '{{ record.campaign.objective }}'
                  - type: AddFields
                    fields:
                      - path:
                          - start_time
                        value: '{{ record.campaign.start_time }}'
    transformations:
      - type: RemoveFields
        field_pointers:
          - - sub_request_status
      - type: AddFields
        fields:
          - path:
              - id
            value: '{{ record.timeseries_stat.id }}'
      - type: AddFields
        fields:
          - path:
              - start_time
            value: '{{ record.timeseries_stat.start_time }}'
      - type: AddFields
        fields:
          - path:
              - granularity
            value: '{{ record.timeseries_stat.granularity }}'
    incremental_sync:
      type: DatetimeBasedCursor
      cursor_field: start_time
      lookback_window: P28D
      cursor_datetime_formats:
        - '%Y-%m-%d'
      datetime_format: '%Y-%m-%d'
      start_datetime:
        type: MinMaxDatetime
        datetime: '{{ config[''start_date''] }}'
        datetime_format: '%Y-%m-%dT%H:%M:%SZ'
      start_time_option:
        type: RequestOption
        field_name: start_time
        inject_into: request_parameter
      end_time_option:
        type: RequestOption
        field_name: end_time
        inject_into: request_parameter
      end_datetime:
        type: MinMaxDatetime
        datetime: '{{ now_utc().strftime(''%Y-%m-%dT%H:%M:%SZ'') }}'
        datetime_format: '%Y-%m-%dT%H:%M:%SZ'
      step: P31D
      cursor_granularity: P1D
spec:
  connection_specification:
    $schema: http://json-schema.org/draft-07/schema#
    type: object
    required:
      - client_id
      - client_secret
      - client_refresh_token
      - action_report_time
      - start_date
      - swipe_up_attribution_window
      - view_attribution_window
    properties:
      client_id:
        type: string
        title: Client ID
        airbyte_secret: true
        order: 0
      client_secret:
        type: string
        title: Client secret
        airbyte_secret: true
        order: 1
      client_refresh_token:
        type: string
        title: Refresh token
        airbyte_secret: true
        order: 2
      action_report_time:
        type: string
        order: 3
        title: Action Report Time
        default: impression
        description: >-
          Specifies the principle for conversion reporting. Could be conversion
          (default), impression**
      end_date:
        type: string
        order: 4
        title: end Date
        format: date
        pattern: ^[0-9]{4}-[0-9]{2}-[0-9]{2}$
        description: end date
      start_date:
        type: string
        title: Start Date
        format: date
        pattern: ^[0-9]{4}-[0-9]{2}-[0-9]{2}$
        order: 5
        description: start date
      swipe_up_attribution_window:
        type: string
        order: 6
        title: Swipe Up Attribution Window
        default: 7_DAY
        description: swipe_up_attribution_window. Could be 1_DAY, 7_DAY(default), 28_DAY
      view_attribution_window:
        type: string
        order: 7
        title: View Attribution Window
        default: 1_DAY
        description: >-
          Attribution window for views. Could be 1_HOUR, 3_HOUR, 6_HOUR, 1_DAY
          (default), 7_DAY, 28_DAY**
    additionalProperties: true
  documentation_url: https://example.org
  type: Spec
metadata:
  autoImportSchema:
    Organization: true
    Adaccounts: true
    Campaigns: true
    Ads: true
    AdaccountsStatsDaily: true
    AdsStatsDaily: true
    CampaignsStatsDaily: true

Abuton · 2023-10-13T20:05:03Z

Abuton
Oct 13, 2023
Author

I'm building this on airbyte version: 0.50.19

I have also tried to use version 0.50.29 and 0.50.30 but I get the same error

Here is the full stack trace error:

Object of type PerPartitionStreamSlice is not JSON serializable - Traceback (most recent call last):

  File "/root/.pyenv/versions/3.9.11/lib/python3.9/site-packages/airbyte_cdk/connector_builder/message_grouper.py", line 236, in _read_stream
    yield from AirbyteEntrypoint(source).read(source.spec(self.logger), config, configured_catalog, {})
  File "/root/.pyenv/versions/3.9.11/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py", line 156, in read
    yield from self.source.read(self.logger, config, catalog, state)
  File "/root/.pyenv/versions/3.9.11/lib/python3.9/site-packages/airbyte_cdk/sources/declarative/manifest_declarative_source.py", line 136, in read
    yield from super().read(logger, config, catalog, state)
  File "/root/.pyenv/versions/3.9.11/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py", line 142, in read
    raise e
  File "/root/.pyenv/versions/3.9.11/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py", line 122, in read
    yield from self._read_stream(
  File "/root/.pyenv/versions/3.9.11/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py", line 197, in _read_stream
    for record in record_iterator:
  File "/root/.pyenv/versions/3.9.11/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py", line 320, in _read_full_refresh
    for _slice in slices:
  File "/root/.pyenv/versions/3.9.11/lib/python3.9/site-packages/airbyte_cdk/sources/declarative/incremental/per_partition_cursor.py", line 132, in stream_slices
    cursor = self._cursor_per_partition.get(self._to_partition_key(partition))
  File "/root/.pyenv/versions/3.9.11/lib/python3.9/site-packages/airbyte_cdk/sources/declarative/incremental/per_partition_cursor.py", line 186, in _to_partition_key
    return self._partition_serializer.to_partition_key(partition)
  File "/root/.pyenv/versions/3.9.11/lib/python3.9/site-packages/airbyte_cdk/sources/declarative/incremental/per_partition_cursor.py", line 24, in to_partition_key
    return json.dumps(to_serialize, indent=None, separators=(",", ":"), sort_keys=True)
  File "/root/.pyenv/versions/3.9.11/lib/python3.9/json/__init__.py", line 234, in dumps
    return cls(
  File "/root/.pyenv/versions/3.9.11/lib/python3.9/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/root/.pyenv/versions/3.9.11/lib/python3.9/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/root/.pyenv/versions/3.9.11/lib/python3.9/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type PerPartitionStreamSlice is not JSON serializable

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Connector Builder: Snapchat Marketing incremental sync error `PerPartitionStreamSlice is not JSON serializable` #31274

Uh oh!

{{title}}

Uh oh!

Replies: 1 comment

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Select a reply

Uh oh!

Connector Builder: Snapchat Marketing incremental sync error PerPartitionStreamSlice is not JSON serializable #31274

Uh oh!

Abuton Oct 11, 2023

Replies: 1 comment

Uh oh!

Uh oh!

Abuton Oct 13, 2023 Author

Connector Builder: Snapchat Marketing incremental sync error `PerPartitionStreamSlice is not JSON serializable` #31274

Abuton
Oct 11, 2023

Abuton
Oct 13, 2023
Author