-
Notifications
You must be signed in to change notification settings - Fork 0
Description
based on a region flag, the brokers may fail over. The splitter service should determine which set of brokers to use based on config (and other dynamic stuff in another story).
The config should look something like this:
output_brokers:
restart_on_config_change: false
target_region: us-east-1
regions:
us-east-1:
- blah
us-west-2:
- blah
On startup, the target_region must match one of the regions in the region map or an error should be returned to uber.fx.
On startup, the target bucket and target region should always be printed with at least level INFO so they are logged in production.
Note that eventually the output_brokers.target_region will be used for manual failover and updated from a centrally located config (this is a separate story in comcast-cl/xmidt - https://github.com/comcast-cl/xmidt/issues/5493)
goshtalt will need to be updated to the latest version so that splitter will pick up the override file. But goshtalt should NOT be updated until this feature is complete, otherwise it will break.
Acceptance Criteria:
wrpkafka brokers are set by above config, rather than the broker config in the publisher section.
goshtalt is updated to the latest version and therefore picks up fies in conf.d to override default behavior