Permission denied (publickey,gssapi-keyex,gssapi-with-mic)

[sasmitamallia31]

The issue is after upgrade for RHEL 8 to 9 or Rhel 9 to 10 via Leapp soultion. We are getting the below error mandatorily for all servers, Hence this should be considered as a known problem.

Permission denied (publickey,gssapi-keyex,gssapi-with-mic)

[sasmitamallia31]

Issue
During in-place upgrades to RHEL 9 and above, upgrade process creates new directives under /etc/ssh/sshd_config.d/.
These directives enforce stricter defaults such as:
PasswordAuthentication no
GSSAPIAuthentication yes
This can block SSH password login after the upgrade.

[jeffmcutter]

This is not done by the Ansible infra.leapp collection. The drop file in question is from the openssh-server package.

If you add a comment to your settings in /etc/ssh/sshd_config it can prevent those from being changed in that file. However, it doesn't help with the the sshd_config.d drop file(s) in the openssh-server package.

I'm testing using 8.10 to 9.6. I don't see PasswordAuthentication in the sshd_config.d drop file. GSSAPIAuthentication is there, but I don't think that should be an issue for logging in. sshd_config also has PasswordAuthentication yes still.

Are you using Password login or SSH keys or?

Are you logging in as root? There is a risk factor high warning about that.

----------------------------------------
Risk Factor: high
Title: Remote root logins globally allowed using password
Summary: RHEL9 no longer allows remote root logins, but the server configuration explicitly overrides this default. The configuration file will not be updated and root is still going to be allowed to login with password. This is not recommended and considered as a security risk.
Remediation: [hint] If you depend on remote root logins using passwords, consider setting up a different user for remote administration. Otherwise you can ignore this message.
Key: e738f78bc8f3a84411a4210e3b609057139d1855
----------------------------------------

It is possible to prevent that from happening by adding a comment on the line:

# grep comment sshd_config
PermitRootLogin yes # The comment will prevent Leapp from changing this setting.
GSSAPIAuthentication no # The comment will prevent Leapp from changing this setting here, but doesn't help with the sshd_config.d drop file which takes precedence.

[sasmitamallia31]

What i am preferring is as below

using "0-custom.conf"
If I create a file named 0-custom.conf as part of the numbering scheme, it should ensure the highest priority & should not delete.

Issue
During in-place upgrades to RHEL 9 and above, upgrade process creates new directives under /etc/ssh/sshd_config.d/.
These directives enforce stricter defaults such as:
PasswordAuthentication no
GSSAPIAuthentication yes
This can block SSH password login after the upgrade.

Intermediate Step (Pre-Upgrade Mitigation)
Create a custom override file /etc/ssh/sshd_config.d/0-custom.conf with the following:
By naming it 0-custom.conf, it loads first in lexicographical order and overrides later conflicting directives.

Final Step (Post-Upgrade Cleanup)
After the upgrade and first reboot, comment out the Include directive in /etc/ssh/sshd_config:
This ensures SSH only uses /etc/ssh/sshd_config and ignores dynamically added directives in .d/

Include /etc/ssh/sshd_config.d/*.conf
PasswordAuthentication yesGSSAPIAuthentication no

Anytime i used to comment it out & leave inside that path it used to override the below during upgrade
Shared below logs

Added by leapp during upgrade from RHEL8 to RHEL9

Include /etc/ssh/sshd_config.d/*.conf

$OpenBSD: sshd_config,v 1.103 2018/04/09 20:41:22 tj Exp $

Added by leapp during upgrade from RHEL8 to RHEL9

Include /etc/ssh/sshd_config.d/*.conf

$OpenBSD: sshd_config,v 1.103 2018/04/09 20:41:22 tj Exp $

This is not done by the Ansible infra.leapp collection. The drop file in question is from the openssh-server package.

If you add a comment to your settings in /etc/ssh/sshd_config it can prevent those from being changed in that file. However, it doesn't help with the the sshd_config.d drop file(s) in the openssh-server package.

I'm testing using 8.10 to 9.6. I don't see PasswordAuthentication in the sshd_config.d drop file. GSSAPIAuthentication is there, but I don't think that should be an issue for logging in. sshd_config also has PasswordAuthentication yes still.

Are you using Password login or SSH keys or?

Are you logging in as root? There is a risk factor high warning about that.

----------------------------------------
Risk Factor: high
Title: Remote root logins globally allowed using password
Summary: RHEL9 no longer allows remote root logins, but the server configuration explicitly overrides this default. The configuration file will not be updated and root is still going to be allowed to login with password. This is not recommended and considered as a security risk.
Remediation: [hint] If you depend on remote root logins using passwords, consider setting up a different user for remote administration. Otherwise you can ignore this message.
Key: e738f78bc8f3a84411a4210e3b609057139d1855
----------------------------------------

It is possible to prevent that from happening by adding a comment on the line:

# grep comment sshd_config
PermitRootLogin yes # The comment will prevent Leapp from changing this setting.
GSSAPIAuthentication no # The comment will prevent Leapp from changing this setting here, but doesn't help with the sshd_config.d drop file which takes precedence.

[richm]

Can we close this? Is the workaround acceptable, or is there something we need to do in the collection?

[jacobklein141]

Can we remove line # 12 " - {key: "PasswordAuthentication", value: "no"}" from leapp_remote_using_root.yml? This just locks out everyone from logging into the box after remediation.

[richm]

Can we remove line # 12 " - {key: "PasswordAuthentication", value: "no"}" from leapp_remote_using_root.yml? This just locks out everyone from logging into the box after remediation.

@bontreger ?

[jacobklein141]

This is not a Leapp inhibitor and there is no need to change this parameter as this locked out even Ansible from executing any further playbook after the remediation changed PasswordAuthentication key to No.

[richm]

This is not a Leapp inhibitor and there is no need to change this parameter as this locked out even Ansible from executing any further playbook after the remediation changed PasswordAuthentication key to No.

Ok. @bontreger is the original author of that remediation, and I would like to know the rationale. In the meantime, I can at least make it optional, and make the default be the current behavior. Would that be acceptable?

Or, something like

# defaults/main.yml
leapp_remediate_ssh_password_auth: "{{ ansible_user == 'root' }}"

and then change leapp_remote_using_root.yml to omit the PasswordAuthentication line when leapp_remediate_ssh_password_auth is false.

That will

• preserve the current behavior for root which IMO is the intention
• allow the playbook to continue for non-root ansible users
• allow the user to control the behavior as desired

@jacobklein141 wdyt?

[jacobklein141]

This will work as a minimum, but ultimately this parameter is not really needed to be part of the remediation. We have successfully upgraded boxes with leapp and "PasswordAuthentication=Yes" and no inhibitors were reported. This parameter needs to be yes for boxes that have users login with just simple password and it's the default for RedHat 8 and 9.

[richm]

This will work as a minimum, but ultimately this parameter is not really needed to be part of the remediation. We have successfully upgraded boxes with leapp and "PasswordAuthentication=Yes" and no inhibitors were reported. This parameter needs to be yes for boxes that have users login with just simple password and it's the default for RedHat 8 and 9.

@jeffmcutter @swapdisk wdyt?

[richm]

@sasmitamallia31 I don't see GSSAPIAuthentication anywhere in this code, so I'm not sure why the collection would be touching that. Maybe it is in leapp itself?

[richm]

@jacobklein141 what about PermitRootLogin prohibit-password, which is the other setting managed by the "remote_using_root" remediation? If you are using Ansible with -u root -k, if that remediation runs, it will lock out root password auth, and therefore Ansible will fail.

[richm]

Also - I really don't like making changes which will make the upgraded system less secure by default. It would be a nasty surprise for someone to upgrade to a newer version of infra.leapp, then run it, then all of a sudden ssh password auth is working, which the admin might not find out about until there is a security breach. IMO that is a much nastier surprise than finding out password auth has stopped working. So I am proposing a new parameter leapp_remediate_ssh_password_auth which will be true by default (i.e. will set PermitRootLogin prohibit-password and PasswordAuthentication no by default), and if you want to opt-out of this behavior, set leapp_remediate_ssh_password_auth: false.

Does this seem reasonable?

[jacobklein141]

That will also work. Thanks, Jacob Klein

…

________________________________ From: Richard Megginson ***@***.***> Sent: Wednesday, February 4, 2026 6:16:32 PM To: redhat-cop/infra.leapp ***@***.***> Cc: Jacob Klein ***@***.***>; Mention ***@***.***> Subject: Re: [redhat-cop/infra.leapp] Permission denied (publickey,gssapi-keyex,gssapi-with-mic) (Issue #268)

________________________________ CAUTION: This email originated from outside of the B&H organization. Do not click on links or open attachments unless you recognize the sender and know that the content is safe.

________________________________ [https://avatars.githubusercontent.com/u/23066?s=20&v=4]richm left a comment (redhat-cop/infra.leapp#268)<https://urldefense.com/v3/__https://github.com/redhat-cop/infra.leapp/issues/268*issuecomment-3850220909__;Iw!!IPNdO0S_!aeyh5KgmlZA722QlJJs_XnmLS9y-ixyiaKoHY0YJNcF0yFgyaIoDTaIyYzGZriscV_wnYxSAap28ONA1JTO6gN8yMQ$> Also - I really don't like making changes which will make the upgraded system less secure by default. It would be a nasty surprise for someone to upgrade to a newer version of infra.leapp, then run it, then all of a sudden ssh password auth is working, which the admin might not find out about until there is a security breach. IMO that is a much nastier surprise than finding out password auth has stopped working. So I am proposing a new parameter leapp_remediate_ssh_password_auth which will be true by default (i.e. will set PermitRootLogin prohibit-password and PasswordAuthentication no by default), and if you want to opt-out of this behavior, set leapp_remediate_ssh_password_auth: false. Does this seem reasonable? — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https://github.com/redhat-cop/infra.leapp/issues/268*issuecomment-3850220909__;Iw!!IPNdO0S_!aeyh5KgmlZA722QlJJs_XnmLS9y-ixyiaKoHY0YJNcF0yFgyaIoDTaIyYzGZriscV_wnYxSAap28ONA1JTO6gN8yMQ$>, or unsubscribe<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/B422AH72XVILVNNLTCEJTWT4KJ4VBAVCNFSM6AAAAACGQNYJVSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTQNJQGIZDAOJQHE__;!!IPNdO0S_!aeyh5KgmlZA722QlJJs_XnmLS9y-ixyiaKoHY0YJNcF0yFgyaIoDTaIyYzGZriscV_wnYxSAap28ONA1JTOO5iu9fQ$>. You are receiving this because you were mentioned.Message ID: ***@***.***>