bitbucketcloud: after push it takes up to a minute to start a pipelinerun

[gerrnot]

We noticed quite a poor performance of incoming webhook events in OpenShift PAC when using bitbucketcloud repositories.

The logs (and user experience) reveals that it takes up to a minute till a PipelineRun ist started.

Here are the relevant logs of the pod openshift-pipelines/pipelines-as-code-controller-*:

2025-11-26T14:13:15.339449497Z {"level":"info","ts":1764166395.3393254,"caller":"bitbucketcloud/bitbucket.go:208","msg":"bitbucket-cloud: initialized client with provided token for user=***masked***"}
2025-11-26T14:14:04.588745629Z {"level":"info","ts":"2025-11-26T14:14:04.588Z","logger":"pipelinesascode","caller":"bitbucketcloud/bitbucket.go:174","msg":"Using PipelineRun definition from source push commit SHA: 07c070e52111edf563407c4f58420965627c6c65","commit":"10dee4fce84b750d7597e61d78a420101ab36d78","provider":"bitbucket-cloud","event-id":"","event-sha":"07c070e52111edf563407c4f58420965627c6c65","event-type":"push","source-repo-url":"https://bitbucket.org/***masked***/***masked***","target-branch":"master","namespace":"***masked***"}

Notice the time delta between the first log line and the second (49 seconds). I assume that internally a clone or pull operation needs to happen in between, but even that terminates locally in about 7 seconds for the given repo.

Exerimenting futher, it seems that the commit count is the one thing that makes this slow.

Squashing that same repo commits I came to the following numbers:

commit count	delay (seconds)
1	1
2333	49

Since we are about to onboard a larger number of business applications where we do not want to lose a large part of the commit history and and the same time guarantee a responsive user experience, this is a significant issue for us, which does not appear in similar CI/CD products.

[zakisk]

@gerrnot Hey, sorry for late reply! in Title you said on push but in description it read We noticed quite a poor performance of incoming webhook events so can you please clarify which one event you meant?
while logs says its push event ""event-type":"push""

[zakisk]

@gerrnot one more question: how is your setup? I guess its on openshift cluster?

[gerrnot]

@zakisk

Yes, push events. So the user git pushes to bitbucket.org. In bitbucket.org, for the repo, a webhook is configured, which points to PAC. In the git repo, there is a file .tekton/cd-pipeline-dev.yaml with the following content:

apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  name: cd-pipeline-dev
  annotations:
    pipelinesascode.tekton.dev/on-event: "[pull_request,push]"
    pipelinesascode.tekton.dev/on-target-branch: "[refs/heads/*,refs/tags/*]"

So a standard git push should trigger pipeline kind of setup.

As indicated, the webhook request part iself seems to behave quickly, since PAC quickly logs something (see description for the logs), but takes very long until the final PR is applied to k8s. So I conclude it must something on PAC side that makes it a slow experience, at least when working with git repos with a high commit count.

For the setup, yes it is OpenShift Dedicated running on GCP.

OpenShift version: 4.19.15

On top of that, we used the Red Hat OpenShift Pipelines Operator to install Openshift pipelines, which shows the following components:

Tekton Pipelines: v1.3.2
Tekton Triggers: v0.33.0
Pipelines as Code: v0.37.3
Tekton Chains: v0.25.1
Tekton Hub (tech-preview): v1.22.3
Tekton Results (tech-preview): v0.16.0
Manual Approval Gate (tech-preview): v0.6.0
Tekton Pruner (tech-preview): v0.2.0

[zakisk]

@gerrnot thanks! I will try to reproduce with this config. one more question I guess you have SSL for git provider webhook payload, right?

[gerrnot]

@zakisk Thanks for looking into it, yes we have configured the webhook on bitbucket.org side with Skip certificate verification = false. But also skipping that does not have an impact on us, so for your reproducer you can also set it to true!