Skip to content

[source-hubspot] Missing properties for Marketing Emails stream for AVRO/Parquet destinations #66479

@kev-datams

Description

@kev-datams

Connector Name

source-hubspot

Connector Version

6.0.1

What step the error happened?

During the sync

Relevant information

Following the v6.0.0 upgrade to migrate Marketing Email stream to the Hubspot API v3, the manifest.yaml has been updated.

Nevertheless, some important fields are missing like:

  • allEmailCampaignIds
  • primaryEmailCampaignId
  • subscriptionName

💡 Even if data is returned in the output JSON and can be stored in _airbyte_additionnal_properties field for JSON storage thanks to additionalProperties: true configuration, this is NOT the case for Avro/Parquet storage.

As explained here and here, the non-declared fields are silently dropped from the output.

🤞 I propose to fix the manifest.yaml to include above fields via this PR : #66480


❓ Bonus: if you have an explanation regarding the below observation, it would be really cool !
The silent dropping seems to be active at "root-level" fields only : here allEmailCampaignIds and primaryEmailCampaignId.
For "child-level" fields, data is still present via nested fields : here subscriptionName available in subscriptionDetails._airbyte_additionnal_properties.

Contribute

  • Yes, I want to contribute

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions