HPA not working as expected in AWS EKS

[ma-kadri]

I have been trying to setup HPA, im on version 0.20.1, This is what my current setup looks like.
This is running in AWS EKS as pods, but for reason it never goes down to 0 pods when there is no load, although when i try it locally on my docker-desktop only a single metrics of TotalNumberOfQueuedAndInProgressWorkflowRuns works fine, and its able to downscale to 0.
please suggest what might be the issue here.

---
apiVersion: actions.summerwind.dev/v1alpha1
kind: HorizontalRunnerAutoscaler
metadata:
  name: k8s-runners-autoscaler
  namespace: actions
spec:
  scaleTargetRef:
    # If we remove the below line, it defaults to RunnerDeployment
    kind: RunnerSet
    # This would be the name of the runner that u set
    # name: k8s-runners
    name: runner-set
  scaleDownDelaySecondsAfterScaleOut: 300
  minReplicas: 0
  maxReplicas: 5
  metrics: 
  - type: PercentageRunnersBusy
    scaleUpThreshold: '0.75'
    scaleDownThreshold: '0.3'
    scaleUpAdjustment: 1
    scaleDownAdjustment: 1
  - type: TotalNumberOfQueuedAndInProgressWorkflowRuns
    repositoryNames:
    - <MY_REPO>
  scaleUpTriggers:
  - githubEvent:
      checkRun:
        types: ["created"]
        status: "queued"
    amount: 1
    duration: "2m"