Worker labels don't run when chaining "self-hosted" and more specific labels

[xrl]

I am trying to create arm64 and amd64 platform-specific runners in order to get fast native builds. When I try to use runs-on: [self-hosted, amd64] the job sits pending forever. When I use just one label it schedules correctly, e.g., runs-on: [amd64].

My runners:

apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: k3-amd64-runner
  namespace: github-actions
spec:
  template:
    spec:
      tolerations:
        - key: purpose
          value: gha
          effect: NoSchedule
      nodeSelector:
        purpose: gha
        arch: amd64
      organization: myorg
      labels: [amd64]
      resources:
        limits:
          cpu: "7.8"
          memory: "14100Mi"
        requests:
          cpu: "7.8"
          memory: "14100Mi"
---
apiVersion: actions.summerwind.dev/v1alpha1
kind: HorizontalRunnerAutoscaler
metadata:
  name: k3-runner-autoscaler
  namespace: github-actions
spec:
  scaleTargetRef:
    name: k3-amd64-runner
  minReplicas: 0
  maxReplicas: 50
  scaleUpTriggers:
    - githubEvent:
        workflowJob: {}
      duration: "30m"

and

apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: k3-arm64-runner
  namespace: github-actions
spec:
  template:
    spec:
      tolerations:
        - key: purpose
          value: gha
          effect: NoSchedule
        - key: arch
          value: arm64
          effect: NoSchedule
      nodeSelector:
        purpose: gha
        arch: arm64
      organization: myorg
      labels: [arm64]
      resources:
        requests:
          cpu: "1.9"
          memory: "14100Mi"
---
apiVersion: actions.summerwind.dev/v1alpha1
kind: HorizontalRunnerAutoscaler
metadata:
  name: k3-arm64-runner-autoscaler
  namespace: github-actions
spec:
  scaleTargetRef:
    name: k3-arm64-runner
  minReplicas: 0
  maxReplicas: 50
  scaleUpTriggers:
    - githubEvent:
        workflowJob: {}
      duration: "30m"

the kube node labels are setup correct, I have those nodegroups in EKS labeled with the correct arch and purpose labels and then I have the taints on there as well.

And this workflow schedules fine:

on: [push]

permissions:
  id-token: write   # This is required for requesting the JWT
  contents: read    # This is required for actions/checkout

jobs:
  build:
    runs-on: ["arm64"]
    steps:
      - name: Git checkout
        uses: actions/checkout@v2

      - name: Check platform
        run: uname -a

but this one waits in the queue forever:

on: [push]

permissions:
  id-token: write   # This is required for requesting the JWT
  contents: read    # This is required for actions/checkout

jobs:
  build:
    runs-on: ["self-hosted", "arm64"]
    steps:
      - name: Git checkout
        uses: actions/checkout@v2

      - name: Check platform
        run: uname -a

and interestingly enough, with just run-on: [self-hosted] the pipeline does not schedule, even with a bunch of idle runners registered to github with the self-hosted label:

[mondo192]

I've encountered this problem as well.

[ramiradaideh]

Are you trying to run your job on 2 different runners at the same time? I dont think that is supported