Sync-Jobs with legacy-encoded passwords

[coldtobi]

Contribution guidelines

• I've read the contribution guidelines and wholeheartedly agree

Checklist prior issue creation

• I understand that failure to follow below instructions may cause this issue to be closed.
• I understand that vague, incomplete or inaccurate information may cause this issue to be closed.
• I understand that this form is intended solely for reporting software bugs and not for support-related inquiries.
• I understand that all responses are voluntary and community-driven, and do not constitute commercial support.
• I confirm that I have reviewed previous issues to ensure this matter has not already been addressed.
• I confirm that my environment meets all prerequisite requirements as specified in the official documentation.

Description

IMAP sync jobs generated via the mailcow API fail when the remote password contains single-byte (non-UTF-8) characters, such as § (0xA7), which are accepted by Exchange 2016 IMAP.

As example, the user has a password ending with "§" (the character 0xA7) , imapsync_runner.pl will encode that as 0xC2 0xA7 in the password file and pass that to imapsync, failing the Authentication.

The failure is caused by forcing :utf8 binmode on the temporary passfile in the imapsync wrapper, which re-encodes the password to UTF-8, changing the byte sequence and breaking authentication. I understand that this is required for paswords that actually contains UTF-8, so while removing the binmode works for me, it will regress on #5528.

I think there is no easy way to correctly fix this, a workaround could be, if the password is UTF-8 and authentication fails to retry with re-encoding the password to latin1. (This seems also a strategy thunderbird employed, at least some years ago -
https://superuser.com/questions/1696433/outlook-password-encoding-for-email-services-imap-smtp )

Steps to reproduce:

1. Setup Sync-Job with Exchange 2016, password contains 8-bit character (like §); options like "--showpasswords" and "--debugimap1" are helpful, as they show how the password is sent over the wire.

2. Sync-Job fails with password error, --showpaswords show that the password is UTF-8 encoded -- see below for extract.

3. manually calling imapsync with password's "§" sent as 0xA7 only.
   imapsync --showpasswords --debugimap1 --dry --justfolders --host1 <redacted> --user1 <redacted> --password1 $(printf"xxxxxxxxxxxx\xa7") --host2 <redacted> --user2 <redacted> --password2 "<redacted>"

4. removing the "binmode( $passfile1, ":utf8" );" line from data/Dockerfiles/dovecot/imapsync_runner.pl -- auth now works.

Logs:

step 2 imapsync debug extract with showpasswords:

	Mail::IMAPClient::__ANON__("3 NO LOGIN failed.\x{d}\x{a}") called at /usr/share/perl5/vendor_perl/Mail/IMAPClient.pm line 1424
	Mail::IMAPClient::_imap_command_do(Mail::IMAPClient=HASH(0x7fef355a4550), "LOGIN <redacted> {14}\x{d}\x{a}xxxxxxxxxxxx\x{c2}\x{a7}") called at /usr/share/perl5/vendor_perl/Mail/IMAPClient.pm line 1248



step 3 manual imapsync invocation:

Connected to <exchange-hostname>
Read: 	* OK The Microsoft Exchange IMAP4 service is ready.
Host1 IP address: 192.168.0.211
Host1 banner: * OK The Microsoft Exchange IMAP4 service is ready.
Sending: 1 CAPABILITY
Sent 14 bytes
Read: 	* CAPABILITY IMAP4 IMAP4rev1 AUTH=PLAIN AUTH=NTLM AUTH=GSSAPI UIDPLUS MOVE ID CHILDREN IDLE NAMESPACE LITERAL+
  	1 OK CAPABILITY completed.
Host1 capability before authentication: IMAP4 IMAP4rev1 AUTH=PLAIN AUTH=NTLM AUTH=GSSAPI UIDPLUS MOVE ID CHILDREN IDLE NAMESPACE LITERAL+ AUTH
Sending literal: 2 LOGIN <redacted> {13}
  	then: xxxxxxxxxxxx�
Sending: 2 LOGIN <redacted> {13}
Sent 33 bytes
Read: 	+ Ready for additional command text.
Sending: xxxxxxxxxxxx�
Sent 15 bytes
Read: 	2 OK LOGIN completed.

Which branch are you using?

master (stable)

Which architecture are you using?

x86_64

Operating System:

Debian GNU/Linux 13 (trixie)

Server/VM specifications:

16 GiB, 8 cores

Is Apparmor, SELinux or similar active?

no

Virtualization technology:

Proxmox

Docker version:

26.1.5+dfsg1

docker-compose version or docker compose version:

Docker Compose version 2.26.1-4

mailcow version:

2025-12a

Reverse proxy:

n/a

Logs of git diff:

no changes to the code.

Logs of iptables -L -vn:

this is not a network issue

Logs of ip6tables -L -vn:

this is not a network issue

Logs of iptables -L -vn -t nat:

this is not a network issue

Logs of ip6tables -L -vn -t nat:

this is not a network issue

DNS check:

this is not a network issue

[milkmaker]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.