Skip to content

Allow ignore unknown value in firehose BQ client #78

@batrov

Description

@batrov

There's issue on starting firehose when proto schema is not matching with BQ table schema. Sample error looks like this:
Provided Schema does not match Table xxx. Field yyy is missing in new schema

Need a feature to ignore this error and fill with default value if proto schema is incomplete.
This can be done by allowing ignoreUnknownValues while instantiating BQ client.
source: https://cloud.google.com/bigquery/docs/reference/rest/v2/tabledata/insertAll#request-body
Firehose version: odpf/firehose:0.7.1

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions