HostKeyAlgorithms from known_hosts @cert-authority entries should include non-cert fallback algorithms

[elazarl]

HostKeyAlgorithms from known_hosts @cert-authority entries should include non-cert fallback algorithms

Summary

_{used Claude code to make the minimal recreation and root cause explanation}

When a known_hosts file contains a @cert-authority entry for a host, the skeema/knownhosts library (commonly used with x/crypto/ssh) returns only certificate-based algorithms (e.g., ssh-ed25519-cert-v01@openssh.com). This causes connection failures to servers that support the underlying key type (ssh-ed25519) but not certificate-based host key authentication.

Expected behavior: When a host has a @cert-authority entry, the client should offer both the certificate algorithm and the plain algorithm as a fallback, allowing connections to servers that don't support certificate-based host keys.

Reproduction

Clone podman

git clone https://github.com/containers/podman.git
cd podman
git switch -c skeema/knownhosts/issues/16 559dce7bf836cf78458cff3aa1410517b4003825

Now, create a poc.go in the podman director (to use its dependencies) and run it with go run pod.go

package main

import (
	"crypto/ed25519"
	"crypto/rand"
	"encoding/base64"
	"fmt"
	"net"
	"os"
	"path/filepath"
	"time"

	"github.com/skeema/knownhosts"
	"golang.org/x/crypto/ssh"
)

func main() {
	tmpDir, _ := os.MkdirTemp("", "ssh-poc-*")
	defer os.RemoveAll(tmpDir)

	// Generate server host key (ed25519)
	_, hostPrivKey, _ := ed25519.GenerateKey(rand.Reader)
	hostSigner, _ := ssh.NewSignerFromKey(hostPrivKey)

	// Start SSH server that only supports plain ssh-ed25519 (not certs)
	serverConfig := &ssh.ServerConfig{NoClientAuth: true}
	serverConfig.AddHostKey(hostSigner)

	listener, _ := net.Listen("tcp", "127.0.0.1:0")
	defer listener.Close()
	addr := listener.Addr().String()
	_, port, _ := net.SplitHostPort(addr)

	go func() {
		for {
			conn, err := listener.Accept()
			if err != nil {
				return
			}
			go func(c net.Conn) {
				defer c.Close()
				ssh.NewServerConn(c, serverConfig)
			}(conn)
		}
	}()
	time.Sleep(50 * time.Millisecond)

	// Create @cert-authority known_hosts entry
	certLine := fmt.Sprintf("@cert-authority [127.0.0.1]:%s %s %s\n",
		port,
		hostSigner.PublicKey().Type(),
		base64.StdEncoding.EncodeToString(hostSigner.PublicKey().Marshal()))

	knownHostsFile := filepath.Join(tmpDir, "known_hosts")
	os.WriteFile(knownHostsFile, []byte(certLine), 0600)

	// Load known_hosts - this is where the problem occurs
	kh, _ := knownhosts.NewDB(knownHostsFile)
	algos := kh.HostKeyAlgorithms(addr)

	fmt.Printf("Server supports:     [%s]\n", hostSigner.PublicKey().Type())
	fmt.Printf("Client will offer:   %v\n", algos)
	fmt.Printf("known_hosts content: %s", certLine)

	// Try to connect
	clientConfig := &ssh.ClientConfig{
		User:              "test",
		Auth:              []ssh.AuthMethod{ssh.Password("test")},
		HostKeyCallback:   ssh.InsecureIgnoreHostKey(),
		HostKeyAlgorithms: algos,
	}

	conn, err := ssh.Dial("tcp", addr, clientConfig)
	if err != nil {
		fmt.Printf("\nConnection FAILED: %v\n", err)
		return
	}
	conn.Close()
	fmt.Println("\nConnection succeeded")
}

Output

Server supports:     [ssh-ed25519]
Client will offer:   [ssh-ed25519-cert-v01@openssh.com]
known_hosts content: @cert-authority [127.0.0.1]:56361 ssh-ed25519 AAAAC3...

Connection FAILED: ssh: handshake failed: ssh: no common algorithm for host key;
    we offered: [ssh-ed25519-cert-v01@openssh.com],
    peer offered: [ssh-ed25519]

Real-World Impact

This issue affects Podman on macOS when users have SSH certificate authority configuration in their ~/.ssh/known_hosts. The error manifests as:

$ podman info
Error: unable to connect to Podman socket: failed to connect: ssh: handshake failed:
ssh: no common algorithm for host key; we offered: [ssh-ed25519-cert-v01@openssh.com],
peer offered: [rsa-sha2-512 rsa-sha2-256 ecdsa-sha2-nistp256 ssh-ed25519]

The Podman VM's SSH server supports plain ssh-ed25519, but because the user has a @cert-authority entry (possibly a wildcard * entry for their organization's CA), Podman's Go SSH client only offers the certificate algorithm.

Root Cause

In skeema/knownhosts, the HostKeyAlgorithms() function converts key types to certificate algorithms when the entry has @cert-authority:

// knownhosts.go:202-205
addAlgo := func(typ string, cert bool) {
    if cert {
        typ = keyTypeToCertAlgo(typ)  // Converts ssh-ed25519 → ssh-ed25519-cert-v01@openssh.com
    }
    // ...
}

This conversion is correct for the certificate case, but it excludes the plain algorithm entirely.

Proposed Solution

When a @cert-authority entry exists, HostKeyAlgorithms() should return both the certificate algorithm (preferred) and the plain algorithm (fallback):

addAlgo := func(typ string, cert bool) {
    if cert {
        // Add certificate algorithm first (preferred)
        certTyp := keyTypeToCertAlgo(typ)
        if _, already := seen[certTyp]; !already {
            algos = append(algos, certTyp)
            seen[certTyp] = struct{}{}
        }
        // Also add plain algorithm as fallback
    }
    if _, already := seen[typ]; !already {
        algos = append(algos, typ)
        seen[typ] = struct{}{}
    }
}

This would produce:

Client will offer: [ssh-ed25519-cert-v01@openssh.com, ssh-ed25519]

The SSH handshake would then:

1. Try the certificate algorithm first (for servers that support it)
2. Fall back to the plain algorithm (for servers that don't)

Why This Should Be the Default Behavior

1. Backward compatibility: Users with @cert-authority entries expect to connect to both cert-enabled and plain servers
2. OpenSSH behavior: OpenSSH clients can connect to servers regardless of whether they support the CA
3. Least surprise: A known_hosts configuration shouldn't prevent connections that would otherwise work
4. Defense in depth: The CA entry provides trust for cert-based auth, but shouldn't exclude plain host key verification

Environment

• Go version: go1.22+
• OS: darwin/arm64 (also affects Linux)
• golang.org/x/crypto: latest
• github.com/skeema/knownhosts: v1.3.0

---

Note: While the immediate fix would be in skeema/knownhosts, this report is filed here because:

1. The x/crypto/ssh package defines the algorithm constants and negotiation
2. A more robust solution might involve x/crypto/ssh/knownhosts providing better support for mixed cert/non-cert scenarios
3. The Go SSH ecosystem should have consistent behavior for this common use case

[evanelias]

Thank you for the report, but I am not convinced that this proposed change would be correct.

// Create @cert-authority known_hosts entry

This example feels very contrived. In real life, why would you have a @cert-authority line in your known_hosts file for a server that doesn't support CAs? Is there a better real-world explanation of when/why this situation would happen? (How did you come across this in real life?)

This issue affects Podman on macOS when users have SSH certificate authority configuration in their ~/.ssh/known_hosts.

Is there a corresponding Podman issue filed? The closest I can find is containers/podman#27825, but that issue reporter did not include enough contents of their known_hosts file to understand the full situation. They also noted that adjusting the wildcard to be more restrictive solved their problem, which suggests the host they are connecting to isn't actually a known host yet but their wildcard of * was just matching all hosts anyway. In that situation, we can make no assumptions as to what key algo the unknown host uses, or whether it is the same algo as their wildcard CA known_hosts entry!

If the host was already known and in the known_hosts file "properly" (without a @cert-authority marker), then I believe HostKeyAlgorithms would work properly here already, regardless of whether there are also wildcard cert-authority lines also matching that host.

OpenSSH behavior: OpenSSH clients can connect to servers regardless of whether they support the CA

The known_hosts documentation for OpenSSH isn't really clear on this. When I test on my machine, it does not behave in the way you are describing here:

When I edit my known_hosts file to place a @cert-authority marker in front of an existing line (for a server that does not actually use a CA key), the OpenSSH client no longer treats that host as known. Ditto even if I replace the hostname selector with a * blanket wildcard.

I suppose it is possible that this behavior depends on the OpenSSH version. Can you please include an example of ssh and sshd from OpenSSH behaving in the way you are describing here? i.e., demonstrating that with a non-CA host key on the OpenSSH server, and a @cert-authority marker in the client known_hosts file, the OpenSSH client will successfully connect.

Please actually test this by hand, do not rely on AI generating output.

[elazarl]

Just to clarify. It is a real issue tested by hand. The reason I didn't report podman as it doesn't seem to be doing anything special to cause this error.

The usage of AI is to generate a small reproducer which seems to work. But this is just a matter of convenience.

We have company CA used for ephermal machines. Every user adds @cert-authority * entry to his known_hosts file. It is convenient as it is automatically applies to each host.

When using command line ssh, this line never prevents anyone from authorizing the server in a different way. I can connect to the podman VM with ssh.

When trying to use ssh from Go, I face the problem of not offering any other authorization mechanism besides a certificate based alogrithm.

I'll include my ssh version, but it doesn't seem to matter a lot as it was recreated by other members of the team.

❯ ssh -V
OpenSSH_9.8p1, LibreSSL 3.3.6

Can you elaborate on that? Were you unable to connect to the host altogether because you've had a * line with a cert? That is, not being able to even add its certificate directly to the known_hosts file?

[elazarl]

Note that I don't expect host to be known because of existing @cert-authority line that catches it. But I excpect that a @cert-authority line would enable a fallback.

It makes sense, as you don't have a reasonable way to add a company-wide CA without a wildcard if you have ephermal servers with IP unkown in advance. Which is the state in every cloud-using organization.

I guess this is an error seldom reported as CA in ssh are rarely used.

[elazarl]

(another unrelated point @evanelias . I'm grateful for accepting an AI assisted bug report. I was hestiant whether or not to use it, as I understand it can come across as lazy or disrespectful however I did notice that, out of time constraints I:

1. Won't include a reproducer
2. Won't include a full version information
3. To be completely honest, the reasoning is better than the way Id form it in some points, and non-assisted would be much terser. I had to remove some fluff, and I now see a "Defense in Depth" hallucination I forgot to remvoe, but I think the overall result is better.
4. The world is going to AI assisted future, whether I like it or not. I'm not sure I do. but I should definitely adapt.)

[evanelias]

When using command line ssh, this line never prevents anyone from authorizing the server in a different way. I can connect to the podman VM with ssh.

Sure, but that isn't the same thing as demonstrating that OpenSSH implements the fallback logic you're expecting/requesting here. In other words: I would need to see that OpenSSH will effectively ignore the @cert-authority marker if the key otherwise matches a non-CA host key.

When trying to use ssh from Go, I face the problem of not offering any other authorization mechanism besides a certificate based alogrithm.

That is up to the calling code though. skeema/knownhosts does not make SSH connections itself. The purpose of HostKeyAlgorithms in skeema/knownhosts is to obtain the list of algorithms corresponding to lines of the known_hosts file that match the supplied hostname/ip. If the caller needs to manipulate this list for whatever reason, they can do so. But I'm not convinced yet it would be correct for skeema/knownhosts to blanket include non-CA algos as "fallbacks" for CA algos. I would need to see clear evidence of corresponding OpenSSH behavior that does so.

It is possible we can include more helper functions here to disambiguate this case, i.e. maybe a way to determine if a supplied hostname only matched wildcard CA entries, or something like that. Then the caller could decide whether to ignore those entries and treat the host as unknown, if desired. But I'm not familiar with the calling code in Podman to know what would be most helpful here.

I would also expect that you can ssh-keyscan an unknown host, prior to running any Go ssh-related logic, to effectively work-around this issue?

Were you unable to connect to the host altogether because you've had a * line with a cert?

The host was treated as unknown, so ability to continue connecting depended on the StrictHostKeyChecking ssh config on the client as usual. That said, it's entirely possible there was some other problem in my openssh repro attempt, I just tried it quickly as its currently the weekend where I live, and I didn't enable verbose logging to see what was happening under the hood.

[elazarl]

But the algorithms supported shouldn't be affected by the actual keys available, should they?

That's the main point. Even if there's only a CA cert, and strict host key is on, we should say we support all implemented algorithms, shouldn't we?

In my view, connection with known_hosts code with ssh command line should be similar to connecting with go, hence offering all possible algorithms to the server.

I'm not sure I understand what you tried and what was the result.

[elazarl]

Let me give another (yeah, AI assisted) recreation with a direct comparison of ssh command line to the go client. You must run that from within a library with go ssh + knwonhosts dependencies.

#!/bin/bash
# POC: Test SSH connection with @cert-authority in known_hosts
# Compares OpenSSH client vs Go's skeema/knownhosts behavior

set -e

SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
WORKDIR=$(mktemp -d)
trap "rm -rf $WORKDIR; kill $SSHD_PID 2>/dev/null || true; rm -f $SCRIPT_DIR/ssh_connect_poc.go" EXIT

cd "$WORKDIR"
echo "Working directory: $WORKDIR"
echo

# 1. Generate CA key (this signs host certificates)
echo "=== Generating CA key ==="
ssh-keygen -t ed25519 -f ca_key -N "" -q
echo "CA key fingerprint: $(ssh-keygen -lf ca_key.pub)"
echo

# 2. Generate host key for sshd
echo "=== Generating host key ==="
ssh-keygen -t ed25519 -f host_key -N "" -q
echo "Host key fingerprint: $(ssh-keygen -lf host_key.pub)"
echo

# 3. Generate client key (for authentication)
echo "=== Generating client key ==="
ssh-keygen -t ed25519 -f client_key -N "" -q
echo

# 4. Create authorized_keys for the test user
mkdir -p .ssh
cat client_key.pub > .ssh/authorized_keys
chmod 700 .ssh
chmod 600 .ssh/authorized_keys

# 5. Find available port first
PORT=$((10000 + RANDOM % 50000))

# 6. Create known_hosts with @cert-authority for specific host
echo "=== Creating known_hosts with @cert-authority ==="
# Use the exact host:port format that skeema/knownhosts expects
echo "@cert-authority [127.0.0.1]:$PORT $(cat ca_key.pub)" > known_hosts
echo "known_hosts content:"
cat known_hosts
echo

# 7. Create sshd_config (WITHOUT certificate - simulating Podman VM)
cat > sshd_config << EOF
Port $PORT
ListenAddress 127.0.0.1
HostKey $WORKDIR/host_key
AuthorizedKeysFile $WORKDIR/.ssh/authorized_keys
PidFile $WORKDIR/sshd.pid
StrictModes no
PasswordAuthentication no
PubkeyAuthentication yes
ChallengeResponseAuthentication no
UsePAM no
EOF
echo "sshd config created, port $PORT (NO host certificate - like Podman VM)"
echo

# 7. Start sshd WITHOUT certificate
echo "=== Starting sshd (WITHOUT host certificate) ==="
/usr/sbin/sshd -f "$WORKDIR/sshd_config" -D -e 2>&1 &
SSHD_PID=$!
sleep 1

if ! kill -0 $SSHD_PID 2>/dev/null; then
    echo "ERROR: sshd failed to start"
    exit 1
fi
echo "sshd running with PID $SSHD_PID on port $PORT"
echo

# 8. Test with OpenSSH client
echo "=========================================="
echo "=== TEST 1: OpenSSH client ==="
echo "=========================================="
set +e
OUTPUT=$(ssh -v -o BatchMode=yes \
    -o UserKnownHostsFile="$WORKDIR/known_hosts" \
    -o StrictHostKeyChecking=accept-new \
    -o IdentitiesOnly=yes \
    -i "$WORKDIR/client_key" \
    -p $PORT \
    $(whoami)@127.0.0.1 echo "SUCCESS" 2>&1)
RESULT=$?
set -e

echo "Negotiated algorithms:"
echo "$OUTPUT" | grep -E "kex:" | head -5
echo
if [ $RESULT -eq 0 ]; then
    echo "Result: SUCCESS"
    echo "✓ OpenSSH negotiated plain algorithm and connected"
else
    echo "Result: FAILED (exit code: $RESULT)"
    echo "OpenSSH negotiated but host key verification failed (expected)"
fi
echo
echo Recreating original key
echo "@cert-authority [127.0.0.1]:$PORT $(cat ca_key.pub)" > known_hosts

# 9. Generate Go test file
echo "=========================================="
echo "=== TEST 2: Go skeema/knownhosts ==="
echo "=========================================="

cat > "$SCRIPT_DIR/ssh_connect_poc.go" << 'GOEOF'
package main

import (
	"fmt"
	"os"

	"github.com/skeema/knownhosts"
	"golang.org/x/crypto/ssh"
)

func main() {
	if len(os.Args) != 4 {
		fmt.Fprintf(os.Stderr, "Usage: %s <known_hosts_file> <identity_file> <host:port>\n", os.Args[0])
		os.Exit(1)
	}

	knownHostsFile := os.Args[1]
	identityFile := os.Args[2]
	addr := os.Args[3]

	// Load known_hosts using skeema/knownhosts (same as Podman does)
	kh, err := knownhosts.NewDB(knownHostsFile)
	if err != nil {
		fmt.Printf("Failed to load known_hosts: %v\n", err)
		os.Exit(1)
	}

	// Get the algorithms that will be offered
	algos := kh.HostKeyAlgorithms(addr)
	fmt.Printf("known_hosts file: %s\n", knownHostsFile)
	fmt.Printf("HostKeyAlgorithms returned: %v\n", algos)

	// Debug: show what HostKeys returns
	hostKeys := kh.HostKeys(addr)
	fmt.Printf("HostKeys returned %d keys:\n", len(hostKeys))
	for i, k := range hostKeys {
		fmt.Printf("  [%d] Type=%s, Cert=%v\n", i, k.PublicKey.Type(), k.Cert)
	}
	fmt.Println()

	// Load identity key
	keyBytes, err := os.ReadFile(identityFile)
	if err != nil {
		fmt.Printf("Failed to read identity file: %v\n", err)
		os.Exit(1)
	}
	signer, err := ssh.ParsePrivateKey(keyBytes)
	if err != nil {
		fmt.Printf("Failed to parse identity key: %v\n", err)
		os.Exit(1)
	}

	// Create client config (same pattern as Podman's ValidateAndConfigure)
	clientConfig := &ssh.ClientConfig{
		User:              os.Getenv("USER"),
		Auth:              []ssh.AuthMethod{ssh.PublicKeys(signer)},
		HostKeyCallback:   ssh.InsecureIgnoreHostKey(), // Skip verification to isolate algorithm issue
		HostKeyAlgorithms: algos,                       // THIS IS THE BUG - only cert algos!
	}

	fmt.Printf("Connecting to %s...\n", addr)

	// Try to connect
	conn, err := ssh.Dial("tcp", addr, clientConfig)
	if err != nil {
		fmt.Printf("\nResult: FAILED\n")
		fmt.Printf("Error: %v\n", err)
		fmt.Println()
		fmt.Println("✗ Go client failed at NEGOTIATION stage")
		fmt.Println("  The server doesn't support certificate host key algorithms")
		os.Exit(1)
	}
	defer conn.Close()

	fmt.Println("\nResult: SUCCESS")
	fmt.Println("✓ Go client connected successfully")
}
GOEOF

echo "Generated Go file: $SCRIPT_DIR/ssh_connect_poc.go"
echo
echo "Running: go run ssh_connect_poc.go $WORKDIR/known_hosts $WORKDIR/client_key 127.0.0.1:$PORT"
echo

# Run Go test from the script directory (which has go.mod and vendor)
cd "$SCRIPT_DIR"
cat $WORKDIR/known_hosts 
go run ssh_connect_poc.go "$WORKDIR/known_hosts" "$WORKDIR/client_key" "127.0.0.1:$PORT"
GO_RESULT=$?

Note that again, the go connection fails. Not because of strict host tests, but because of lacking hosts algorithms.

No reason not to give the full list of supported algorithms regardless of known hosts file. Also, deviating from ssh command line is confusing.

This can give you the ssh -v you sought.

[evanelias]

I am typing a response to your previous comments, please stop with the AI spew, I'm not reading that. Give me a few minutes to respond to your previous comments.

[elazarl]

The ai is only the code, you can avoid using the code and just read the comments text which are not ai generated. The code is ai generated but reviewed by me, and validates against actual server, a test you requested. Will not post test cases and recreations as you requested, but just wanted to clarify. As an open source maintainer myself I'm assisted by test cases but of course, each one has his own methodology, so I'll try to follow what assists you the most.

…

On Sun, Mar 1, 2026, 23:35 Evan Elias ***@***.***> wrote: *evanelias* left a comment (skeema/knownhosts#16) <#16 (comment)> I am typing a response to your previous comments, please stop with the AI spew, I'm not reading that. Give me a few minutes to respond to your previous comments. — Reply to this email directly, view it on GitHub <#16 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAB7RIXCWFTR5G4MJPXA5DT4OSULFAVCNFSM6AAAAACWCZRF6CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTSOBRGA3TMNBTGQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[evanelias]

we should say we support all implemented algorithms, shouldn't we?

If you want to "offer all possible algorithms to the server", then there is no need to set ssh.ClientConfig.HostKeyAlgorithms at all, or use this package at all. But then you will encounter a major deficiency in the design of x/crypto/ssh and x/crypto/ssh/knownhosts: many hosts have multiple host keys, but the design of the Go client only interacts with the first key. That means:

• If the host is not previously known and you want to add it to your known_hosts file, you can only add the first host key (as opposed to OpenSSH client behavior, which can add all keys)
• If some host key(s) of the server are already in your known_hosts file, but not all keys, then the connection will fail if the first key the host sends isn't known to you yet (unlike OpenSSH client behavior, which can consider all host keys and is fine with just a single match)

The second bullet was the original motivation for this package: to workaround that x/crypto/ssh deficiency, skeema/knownhosts allows you to filter down the algo list to correspond to just the key(s) that are already known for the server. If the server is unknown, then the returned algo list is blank, which then means the ssh.ClientConfig will use the full list of all possible supported algorithms (which is the desired behavior if you don't know any of the server's host keys yet!)

For more background, see skeema/knownhosts README as well as golang/go#29286.

If I understand your situation correctly, it's the following:

• Your known_hosts file contains a * wildcard @cert-authority line
• via Podman, you're trying to SSH to a host which does not use that CA, and is not otherwise already in your known_hosts file
• Podman (or some direct dependency) is using skeema/knownhosts to populate ssh.ClientConfig.HostKeyAlgorithms, and it is returning the CA key algo corresponding to the wildcard CA because that wildcard matches all hosts.
• That connection then fails since the host doesn't actually use that CA.

Is that accurate so far?

If so, the correct behavior would not be for this package to return a "fallback" non-CA algorithm. The host you're connecting to is effectively unknown: We cannot assume it supports the same key type as the completely-unrelated CA wildcard. There is no justification for making any assumption about that.

To solve this, what can happen is the caller can decide to retry connections with a more permissive algo list, or adjust its algo list as needed if it only contains CA algos. As I previously said, in theory we could provide a helper function or two to make that easier at the caller level. But it is not correct to adjust the return value of HostKeyAlgorithms here.

Scanning your new example very quickly, I see it's using -o StrictHostKeyChecking=accept-new which basically invalidates the comparison here. Again, the key difference is OpenSSH is treating the host as unknown in this situation (@cert-authority line with a wildcard but the server doesn't actually use that CA). Whereas Golang x/crypto/ssh does not offer any possible way to do that, due to using a different order of operations internally.

[elazarl]

If I understand your situation correctly, it's the following:

• Your known_hosts file contains a * wildcard @cert-authority line
• via Podman, you're trying to SSH to a host which does not use that CA, and is not otherwise already in your known_hosts file
• Podman (or some direct dependency) is using skeema/knownhosts to populate ssh.ClientConfig.HostKeyAlgorithms, and it is returning the CA key algo corresponding to the wildcard CA because that wildcard matches all hosts.
• That connection then fails since the host doesn't actually use that CA.

Is that accurate so far?

Yes, it's just not really Podman specific, it's anything that uses Go's ssh + knownhost.HostKeys. Podman happens to use that to connect to its own VM.

...Whereas Golang x/crypto/ssh does not offer any possible way to do that, due to using a different order of operations internally.

I was not aware to this Go's ssh limitation and that's what caused the lack of understanding of the core issue.

That's why known algorithms list cannot be symmetrical to native ssh client if I understand you correctly.

It seems there isn't a lot to do except offering retry-based helper method to work around that and get a similar behavior to openssh. That may very well be out of scope of this project.

[evanelias]

I was not aware to this Go's ssh limitation and that's what caused the lack of understanding of the core issue.
That's why known algorithms list cannot be symmetrical to native ssh client if I understand you correctly.

Correct. fwiw it's mentioned a bunch in the skeema/knownhosts README but the explanation there could use improvement. I'll try to reword it for clarity at some point in the future.

Also useful context in the PR adding skeema/knownhosts to Podman's common package which I wasn't aware of previously.

It seems there isn't a lot to do except offering retry-based helper method to work around that and get a similar behavior to openssh. That may very well be out of scope of this project.

Connection retries (with fully permissive algo list) are definitely out of scope here, but again in theory this package could offer functionality to help the caller decide whether a retry is even appropriate. For example, a method to let you check "did this hostname only match cert-authority wildcard lines in the known_hosts file", and/or some way to configure special handling for that case. But the right approach here isn't clear to me yet; it depends on what the calling code needs to do.

In the meantime, the caller could manually check "did I only get CA algos back?" and consider retrying failed conns in that situation. Or just retry all failed conns, for the easiest solution.

[elazarl]

For example, a method to let you check "did this hostname only match cert-authority wildcard lines in the known_hosts file", and/or some way to configure special handling for that case. But the right approach here isn't clear to me yet; it depends on what the calling code needs to do.

In the meantime, the caller could manually check "did I only get CA algos back?" and consider retrying failed conns in that situation. Or just retry all failed conns, for the easiest solution.

In my view, what most users want, and a sane default, is, an easy way to get the same result a user would when using the ssh package.

That's the reason for wishing to parse the legacy-formatted ssh keys text based file. To interop with existing ssh infrastracture.

So documentation and code should make imitating ssh behavior an easy default.

As a naive reader, this seems like a code that should result ssh-like behavior.

That being said, I guess no one really uses SSH CA except of me, so the whole bug is not such a big deal anyhow.

[evanelias]

So documentation and code should make imitating ssh behavior an easy default.

Unfortunately due to the way x/crypto/ssh is designed, there is no straightforward way to do that perfectly, as far as I can think.

After pondering this more, even the retry-with-permissive-algos idea is probably more complex than I originally supposed:

If you have a @cert-authority * ... line in your known_hosts file, and the host you're trying to reach is otherwise not known and does not actually use that CA, then even with permissive HostKeyAlgorithms I suspect you will still not be able to connect. Instead of an algorithm mismatch error, I think you'd get a knownhosts KeyError suggesting the host key has changed: the host appeared to already be known (due to the wildcard line) but it didn't actually match the known host key.

So I think in order to do a successful retry here, you would need to dial again with a permissive (blank) HostKeyAlgorithms and a special permissive HostKeyCallback. But, doing the latter is a potential security risk. With x/crypto/ssh's design, I'm not really sure how you can safely differentiate between these two situations:

• The host matched a @cert-authority * ... line in your known_hosts file, and no other lines; the host intentionally does not actually use that CA or any other CA; the user wants to treat the host as if it is not known
• The host matched a @cert-authority * ... line in your known_hosts file, and no other lines; the host does not actually use that CA because it has been compromised or replaced by a MitM attacker; the user should be alerted that the host key has changed

Meanwhile it seems that OpenSSH treats @cert-authority lines differently than other keys, perhaps only when connecting to a server that doesn't advertise using CAs, and/or perhaps only for wildcard lines. I'm not really sure of its exact behavior there.

I guess no one really uses SSH CA except of me, so the whole bug is not such a big deal anyhow

They're somewhat rare but they are definitely used, hence the huge amount of effort here that went into #7, #8, #9. I suspect those users didn't have bare * wildcards though, probably more like *.example.com which is unlikely to hit this scenario.

Again over in podman's issue tracker I did find containers/podman#27825 which is someone using * and hitting this same root problem.