-
Notifications
You must be signed in to change notification settings - Fork 30
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Currently we have code like
autoCreateTables = config.getBoolean(BigQuerySinkConfig.TABLE_CREATE_CONFIG);
upsertDelete = config.getBoolean(BigQuerySinkConfig.UPSERT_ENABLED_CONFIG)
|| config.getBoolean(BigQuerySinkConfig.DELETE_ENABLED_CONFIG);
useCredentialsProjectId = config.getBoolean(BigQuerySinkConfig.USE_CREDENTIALS_PROJECT_ID_CONFIG);
useStorageApi = config.getBoolean(BigQuerySinkConfig.USE_STORAGE_WRITE_API_CONFIG);
useStorageApiBatchMode = useStorageApi && config.getBoolean(BigQuerySinkConfig.ENABLE_BATCH_MODE_CONFIG);
retry = config.getInt(BigQuerySinkConfig.BIGQUERY_RETRY_CONFIG);
retryWait = config.getLong(BigQuerySinkConfig.BIGQUERY_RETRY_WAIT_CONFIG);
allowNewBigQueryFields = config.getBoolean(BigQuerySinkConfig.ALLOW_NEW_BIGQUERY_FIELDS_CONFIG);
allowRequiredFieldRelaxation = config.getBoolean(BigQuerySinkConfig.ALLOW_BIGQUERY_REQUIRED_FIELD_RELAXATION_CONFIG);
Where the configuration logic is scattered across the system in various places.
These should be implemented within the configuration class.
autoCreateTables = config.autoCreateTables();
upsertDelete = config.upsertDelete();
useCredentialsProjectId = config.useCredentialsFromProjectId();
useStorageApi = config.useStorageApi();
useStorageApiBatchMode = config.useStorageApiBatchMode();
RetryConfig retryCfg = config.getRetryConfig();
allowNewBigQueryFields = config.allowNewBigQueryFields();
allowRequiredFieldRelaxation = config.allowRequiredFieldRelaxation();
Where RetryConfig is
Interface RetryConfig {
int getCount();
long getDelay()
}
That will encapsulate the conversions and logic gymnastics into the config class keep the code cleaner.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request