pull: "No valid credentials" when using an ssh-agent

[mirkolenz]

Bug Report

pull: "no valid credentials" when using ssh-agent

Description

As outlined in #215, I tried setting up SSH keys using the webfactory/ssh-agent action. However, dvc always complained that no valid credentials were provided even though the SSH keys were added as deploy keys to the individual repositories. This is the same action that was mentioned in treeverse/dvc#7702 as well, so I see some similarities here (even though the linked issue only mentions these problems on Windows-based machines).

Reproduce

Use the following GitHub action:

on:
  push:
    branches:
      - main
      - beta
env:
  POETRY_VERSION: 1.3.1
  PYTHON_VERSION: 3.9
  DVC_VERSION: 2.43.0
jobs:
  dvc-test:
    runs-on: ubuntu-latest
    steps:
      - uses: webfactory/ssh-agent@v0.7.0
        with:
          ssh-private-key: |
            ${{ secrets.SSH_ARGUEBASE_PUBLIC }}
            ${{ secrets.SSH_ARGUEBASE_PRIVATE }}
      - uses: actions/checkout@v3
      - uses: iterative/setup-dvc@v1
        with:
          version: ${{ env.DVC_VERSION }}
      - run: dvc pull --force --verbose

Expected

DVC uses the credentials provided by the ssh-agent and pulls the data. However, dvc always complains that no valid credentials were provided.

Environment information

The problem occurs on GitHub actions using ubuntu-latest and the setup-dvc action.

Additional Information (if any):

I ran the following script provided by @dtrifiro in the same GitHub action:

import asyncio

import asyncssh


async def main():
    async with asyncssh.agent.connect_agent() as agent:
        keys = await agent.get_keys()
        for key in keys:
            print(key.algorithm, key.get_comment())


if __name__ == "__main__":
    asyncio.run(main())

and got this output:

python dvc_test.py
  shell: /usr/bin/bash -e {0}
  env:
    POETRY_VERSION: 1.3.1
    PYTHON_VERSION: 3.9
    DVC_VERSION: 2.43.0
    SSH_AUTH_SOCK: /tmp/ssh-XXXXXXiyBBBE/agent.1658
    SSH_AGENT_PID: 1659
    pythonLocation: /opt/hostedtoolcache/Python/3.9.16/x64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.9.16/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.9.16/x64
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.9.16/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.9.16/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.9.16/x64/lib
b'ssh-ed25519' git@github.com:recap-utr/arguebase-public.git
b'ssh-ed25519' git@github.com:recap-utr/arguebase-private.git

meaning that the keys are picked up by asyncssh.

[pmrowla]

I can't reproduce this issue with the latest DVC release (pmrowla/test-dvc-ssh#1)

@mirkolenz are you still experiencing this problem?

[mirkolenz]

Thank you for the investigation! I just tested it again (i.e., I disabled my mitigations for this issue) and it is still relevant. However, I now get a different error: Instead of complaining about missing credentials, dvc begins pulling the repo and just stalls. I had to manually abort the action after 10 minutes. Here are the relevant files:

• Workflow run
• GitHub workflow

I hope this is helpful for you to triage this error. Thanks again for looking at this issue 😃

[mirkolenz]

For reference: Here is the workflow with mitigations in place

[pmrowla]

Hard to tell what might be hanging, it's possible the recent scmrepo changes may address this but we'll have to wait for the next DVC release to check. I'll ping you when there's a new release to test against.

[pmrowla]

@mirkolenz can you please try the latest DVC release

[mirkolenz]

Thanks for coming back to this topic! I am quite busy at the moment as two deadlines are approaching. I will try to provide you with an update sometime this week.

[mirkolenz]

Sorry for the delay. I tested my workflow with v2.54.0 and v2.55.0. Both show the same behavior: The job is running endlessly and is canceled by GitHub at some point.