Reduce default concurrency in config.example.yml#775
Conversation
The default concurrency settings should be conservative and increased when there's a need rather than be high by default and scaled down when they cause problems.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #775 +/- ##
==========================================
- Coverage 80.05% 80.03% -0.03%
==========================================
Files 134 134
Lines 18834 18834
Branches 2800 2800
==========================================
- Hits 15078 15073 -5
- Misses 2912 2916 +4
- Partials 844 845 +1 |
|
This won't actually fix the problem, as they are hard-coded elsewhere. We can look into reducing them, I agree they are way too high, but I disagree with setting them to 1. If a user produces enough events to ever use more than a single parallel write, they probably want to have at least a few parallel writes. It's not always easy to understand that the extractor isn't keeping up, and users do not read documentation for extractors. This is not the same as API docs or most of our other technical documentation, as the level of technical know-how is often way lower with extractors. I've personally never seen a user correctly change these settings. I've seen them set to 0, and to 1000. If we can set sane defaults that rarely need changing I would much rather do that. |
The default concurrency settings should be conservative and increased when there's a need rather than be high by default and scaled down when they cause problems.