Actions Metrics Server on GHES - no metrics

[stelligent-joao]

I am having issues getting any sort of data from the new actions metrics server on GHES 3.7.4.
Upon deploying the new actions metrics server using the config bl

githubEnterpriseServerURL: https://github.akgj.com/api/v3
authSecret:
  github_app_id: <hidden>
  github_app_installation_id: <hidden>
  github_app_private_key: <hidden>
certManagerEnabled: true
scope:
  singleNamespace: false
githubWebhookServer:
  secret:
    enabled: true
    name: "github-webhook-server"
  enabled: true
  service:
    type: LoadBalancer
    ports:
      - port: 443
        targetPort: http
        protocol: TCP
        name: http
        nodePort: 31017
actionsMetricsServer:
  imagePullSecrets: []
  logLevel: "-4"
  enabled: true
  labels: {}
  replicaCount: 1
  secret:
    enabled: true
    create: false
    name: "github-webhook-server"
  serviceAccount:
    create: true
  service:
    type: ClusterIP
    ports:
      - port: 80
        targetPort: http
        protocol: TCP
        name: http
actionsMetrics:
  serviceMonitor: true
  port: 8009
  proxy:
    enabled: false
  serviceMonitorLabels: {release: prometheus-stack}
metrics:
  serviceMonitor: true
  port: 8002
  serviceMonitorLabels: {release: prometheus-stack}
  proxy:
    enabled: false

The metrics server logs don't ever log any activity upon start

kubectl logs -n actions-runner-system actions-runner-controller-actions-metrics-server-fdcc6ffd-5w4tv -f
2023-01-27T19:39:36Z	INFO	Using the value from GITHUB_WEBHOOK_SECRET_TOKEN for -github-webhook-secret-token

curling the actions-runner-controller-metrics-service exposes many metrics but non of the new "actions metrics server" even after running workflows in github that trigger scale up actions and those events can be seen in the github webhook server logs.

Has anyone managed to get this working on GHES hosted?

[dhawalseth]

@stelligent-joao have you setup webhooks? Please checkout #2231

[dhawalseth]

Refer to this doc which talks about creating webhooks for github-webhook-server but can be referenced for actions-metrics-server as it works the same way.
https://github.com/actions/actions-runner-controller/blob/master/docs/automatically-scaling-runners.md

[stelligent-joao]

Yes, so I have updated the config so that the actionsmetricsserver uses a loadbalancer and I have configured my github app to point to that service load balancer. I can see on the actions-runner-controller-actions-metrics-server-86b5bbd59lcd8z pod logs that the webhooks are hitting the service.

I can see the actions-runner-controller-actions-metrics-serve-service-monitor is being deployed however when I curl the metrics service, the new metrics are not displaying

authSecret:
  github_app_id: <hidden>
  github_app_installation_id: <hidden>
  github_app_private_key: <hidden>
certManagerEnabled: true
scope:
  singleNamespace: false
actionsMetricsServer:
  enabled: true
  secret:
    enabled: true
    create: false
    name: "github-webhook-server"
#     ### GitHub Webhook Configuration
    github_webhook_secret_token: <hidden>
  service:
    type: LoadBalancer
    ports:
      - port: 443
        targetPort: http
        protocol: TCP
        name: http
actionsMetrics:
  serviceMonitor: true
  port: 8009
  proxy:
    enabled: false
  serviceMonitorLabels: {}
metrics:
  proxy:
    enabled: false

    
   


helm git:(main) ✗ kubectl logs -n actions-runner-system actions-runner-controller-actions-metrics-server-86b5bbd59lcd8z
2023-02-14T13:49:53Z	INFO	Using the value from GITHUB_WEBHOOK_SECRET_TOKEN for -github-webhook-secret-token
2023-02-14T13:58:56Z	INFO	workflowjobmetrics-webhookserver	ok	{"event": "workflow_job", "hookID": "112", "delivery": "b8ac99a0-ac6f-11ed-9e5f-08bd72101be8"}

[stelligent-joao]

I believe it's related to issue #2280