Max resources limit reached when high throughput

[AgnieszkaKlobus12]

Bug Report

Describe the bug
During load tests in our system Fluentbit was not able to process data and send it to opentelemetry output.

To Reproduce
I prepared two small python files for ease of testing.

Run http server local:
http-server.py

Run fluentbit with config:

service:
  log_level: debug
  http_server: on
  health_check: on
  http_port: 2020

pipeline:
  inputs:
    - name: opentelemetry
      port: 4319

  outputs:
    - name: opentelemetry
      retry_limit: 1
      match: '*'
      host: localhost
      port: 4318

Send logs using pyton script. One run should be enough, can run it twice for good measure.
send_dummy.py

Expected behavior
Logs are expected to be send to opentelemetry output and received on http server, but instead "max resources limit reached" error is logged and no logs are send.

Logs
Full log from start to terminating process:

Fluent Bit v4.1.0
* Copyright (C) 2015-2025 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

______ _                  _    ______ _ _             ___   __  
|  ___| |                | |   | ___ (_) |           /   | /  | 
| |_  | |_   _  ___ _ __ | |_  | |_/ /_| |_  __   __/ /| | `| | 
|  _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / /_| |  | | 
| |   | | |_| |  __/ | | | |_  | |_/ / | |_   \ V /\___  |__| |_
\_|   |_|\__,_|\___|_| |_|\__| \____/|_|\__|   \_/     |_(_)___/


[2025/09/24 08:37:10.579179000] [ info] [fluent bit] version=4.1.0, commit=75a517bfe0, pid=27648
[2025/09/24 08:37:10.579436000] [ info] [storage] ver=1.5.3, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2025/09/24 08:37:10.579447000] [ info] [simd    ] disabled
[2025/09/24 08:37:10.579451000] [ info] [cmetrics] version=1.0.5
[2025/09/24 08:37:10.579457000] [ info] [ctraces ] version=0.6.6
[2025/09/24 08:37:10.579615000] [ info] [input:opentelemetry:opentelemetry.0] initializing
[2025/09/24 08:37:10.579619000] [ info] [input:opentelemetry:opentelemetry.0] storage_strategy='memory' (memory only)
[2025/09/24 08:37:10.579703000] [ info] [input:opentelemetry:opentelemetry.0] listening on 0.0.0.0:4319
[2025/09/24 08:37:10.580067000] [ info] [http_server] listen iface=0.0.0.0 tcp_port=2020
[2025/09/24 08:37:10.580082000] [ info] [sp] stream processor started
[2025/09/24 08:37:10.580127000] [ info] [engine] Shutdown Grace Period=5, Shutdown Input Grace Period=2
[2025/09/24 08:37:16.585125000] [error] [output:opentelemetry:opentelemetry.0] max resources limit reached
[2025/09/24 08:37:17.584641000] [error] [output:opentelemetry:opentelemetry.0] max resources limit reached
^C[2025/09/24 08:37:43] [engine] caught signal (SIGINT)

Your Environment

• Version used: 4.1.0 (build locally from master)

Additional context
We expect fluentbit to be able to handle lots of data, coming with rate of over 100 logs per second. How can this be achieved? Or is this error the expected behavior?

[edsiper]

Current opentelemetry output plugin has a hard limit of 100 resources within the same buffer.

This is going to change in future versions, I suggest to adjust the python script.