"queue buffer size exhausted" error with only 31 projects in the configuration?

[kastl-ars]

Dear all,

there are multiple issues regarding the "queue buffer size exhausted" error.
#926, #908, #648 and #327

I tried to follow the recommendations in our setup and reduced the number of projects in the instance to 31.

However, every 5 minutes I get approx. 8700 lines of this error in the pod's log. even with only one project.

Our configuration is relatively short:

apiVersion: v1
data:
  config.yml: |
    log:
      format: json
    gitlab:
      url: https://gitlab.example.org/
      enable_health_check: true
      enable_tls_verify: true
      maximum_requests_per_second: 5
      maximum_jobs_queue_size: 1000
    projects: 
      - name: foo/bar

There is no wildcard setting, but I suspect the following line from the pod's logs could give a hint:

{"level":"info","msg":"pull projects from wildcards","on-init":"yes","scheduled":"every 1800s","time":"2025-08-25T12:18:47Z"}

I tried disabling the pull from wildcards, but still I get this huge amount of errors. The setting is being respected:

{"level":"info","msg":"pull projects from wildcards","on-init":"no","scheduled":"no","time":"2025-08-25T12:24:24Z"}

Is this because there are some metrics already exported, which it then tries to continue exporting, even though the project is no longer in the list of projects?

Kind Regards,
Johannes

[kastl-ars]

Hmmm, does garbage-collect only remove the "unwanted" projects from the queue or does it delete the metrics, too?

https://github.com/mvisonneau/gitlab-ci-pipelines-exporter/blob/main/docs/configuration_syntax.md?plain=1#L164

[kastl-ars]

Ideas, anyone?

[github-actions]

This issue has been inactive for and is scheduled to close in days.