Skip to content

Reduce default concurrency in config.example.yml#775

Open
amorken wants to merge 1 commit intomasterfrom
amorken-concurrency-limits
Open

Reduce default concurrency in config.example.yml#775
amorken wants to merge 1 commit intomasterfrom
amorken-concurrency-limits

Conversation

@amorken
Copy link
Copy Markdown

@amorken amorken commented Jan 30, 2025

The default concurrency settings should be conservative and increased when there's a need rather than be high by default and scaled down when they cause problems.

The default concurrency settings should be conservative and increased when there's a need rather than be high by default and scaled down when they cause problems.
@codecov
Copy link
Copy Markdown

codecov bot commented Jan 30, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.03%. Comparing base (a02b9cf) to head (7ad9163).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #775      +/-   ##
==========================================
- Coverage   80.05%   80.03%   -0.03%     
==========================================
  Files         134      134              
  Lines       18834    18834              
  Branches     2800     2800              
==========================================
- Hits        15078    15073       -5     
- Misses       2912     2916       +4     
- Partials      844      845       +1     

see 5 files with indirect coverage changes

@einarmo
Copy link
Copy Markdown
Contributor

einarmo commented Jan 30, 2025

This won't actually fix the problem, as they are hard-coded elsewhere. We can look into reducing them, I agree they are way too high, but I disagree with setting them to 1. If a user produces enough events to ever use more than a single parallel write, they probably want to have at least a few parallel writes.

It's not always easy to understand that the extractor isn't keeping up, and users do not read documentation for extractors. This is not the same as API docs or most of our other technical documentation, as the level of technical know-how is often way lower with extractors.

I've personally never seen a user correctly change these settings. I've seen them set to 0, and to 1000. If we can set sane defaults that rarely need changing I would much rather do that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants