Extract DataStreamSpec and move configs to separate structs#12
Extract DataStreamSpec and move configs to separate structs#12realglebivanov wants to merge 5 commits intoecomtech-oss:mainfrom
Conversation
realglebivanov
force-pushed
the
scoped_config
branch
3 times, most recently
from
9aa3db7 to
95df513
Compare
lib/kafka_batcher/accumulator.ex
Outdated
| Accumulator, | ||
| Accumulator.State, | ||
| MessageObject, | ||
| PipelineUnit, |
There was a problem hiding this comment.
I named it PipelineUnit because this struct represents the unit of work that drives a single batching pipeline - it bundles together all of the configs relevant for one topic/partition pipeline (collector, accumulator config, producer config, etc.).
It’s meant to be the single coherent entity that the supervisor and worker processes use to start and configure all the pieces involved in a pipeline.
There was a problem hiding this comment.
I suggest calling the ConfigEntity or maybe ConfigUnit, if the entity is intended for config storage
There was a problem hiding this comment.
I found options like DataFlowSpec, StreamConfig, DataStreamSpec
What's your opinion?
There was a problem hiding this comment.
I think, DataStreamSpec is sound more relevant for this entity.
realglebivanov
force-pushed
the
scoped_config
branch
from
95df513 to
99ea0a9
Compare
realglebivanov
changed the title
realglebivanov
changed the title
realglebivanov
force-pushed
the
scoped_config
branch
from
ed329b7 to
d9ae0d7
Compare
|
@romul Please, have a look at this draft |
realglebivanov
force-pushed
the
scoped_config
branch
from
6ddc5bb to
bfdee3a
Compare
| max_batch_bytesize: config.max_batch_bytesize, | ||
| max_accumulator_restarts: config.max_accumulator_restarts, | ||
| accumulator_mod: config.accumulator_mod | ||
| ] |
There was a problem hiding this comment.
It could be just config |> Enum.into([])
3 participants
No description provided.