Skip to content

log-ingestor: Group ingestion job identifier fields when detecting key prefix conflicts. #1805

@LinZhihao-723

Description

@LinZhihao-723

Request

The current log-ingestor verifies whether the key-prefix has conflict by checking against several ingestion job config fields explicitly (dataset, bucket name, etc).
However, this is not scalable:

  • Validating things field by field is too verbose.
  • The similar identifier (fields) will be used later when we need to share the buffer.

Possible implementation

We should create a struct to hold all identifier fields and implement Eq method for it.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions