Remove protected chats #7116

link2xt · 2025-08-16T02:26:16Z

Based on #7147

Since key-contacts cannot change their keys and having "verification" is less important for contact identification than having a chat history or shared chats with a contact, UIs have stopped displaying green checkmarks everywhere (deltachat/deltachat-android#3828).

There was also a bug #7107 (closed in #7113) that resulted in updating "verified by" information for already verified contacts, so the information about "verifiers" is lost.

We also want to make "verification" more meaningful by only verifying contacts if contact QR code is scanned directly within a short period of time and not just because someone used an invite link to join the group: #7111

Because of this it makes sense to reset existing verification status for contacts and start from scratch.

We can also allow to remove verification from contacts (https://support.delta.chat/t/how-would-i-remove-the-verified-checkmarks-from-one-invite-code-in-retrospect/3403) as it will not break "protected" chats anymore, but this is out of scope for this PR.

The plan is to stop creating new "protected" chats, convert already "protected" group chats into just group chats, reset existing verifications and switch to a new mechanism for verification gossip so old verifications don't resurrect via gossip.

Based on #7146. I have moved easy to review changes there.

What this PR does:

Remove is_protection_broken APIs. (moved to Various protected group changes #7146)
Remove is_profile_verified API. (moved to Various protected group changes #7146)
Stop indirectly verifying contacts via messages with a Chat-Verified: 1 header.
Add new _verified attribute for Autocrypt-Gossip and use it to indirectly verify key-contacts when such attribute is received from a verified contact (whether directly verified or not).
Make sure that direct verifications cannot happen via "gossip" from self via outgoing messages sent by other devices. This is another reason to reset existing verifications, apparently existing "direct" verifications may be not actually direct but gossiped by own devices.
Remove error paths from add_parts that result in "The message was sent by non-verified contact" messages. We likely don't need to pass verified_encryption around, it is only used to decide whether we want to accept verification gossip from the contact.
Remove Chat.is_protected() APIs.
Remove API to create new protected groups
Update dc_contact_is_verified documentation.

To close #7111 we also need to:

Expiring QR codes? Not sure we want to do this in the same PR. It will make verifications more meaningful though, so ideally should be done now too.
Reset existing verifications via migration. We should do this at the same time as we switch to the new gossiping mechanism, so it should be part of this PR.

Closes #7080 (replace verification gossip mechanism with a new one that works using Autocrypt-Gossip header and is independent of "protected chats")
Closes #7112 (removing deprecated and unneeded APIs)

src/mimeparser.rs

link2xt · 2025-08-16T02:34:46Z

src/receive_imf.rs

    mimeparser: &MimeMessage,
 ) -> Result<()> {
-    if mimeparser.get_header(HeaderDef::ChatVerified).is_none() {


Old Chat-Verified mechanism is completely ignored here.
All the tests now work using only the new mechanism.

Did you test how group members using the old&new version interact? E.g., would be bad if the old version rejects messages sent by new Delta Chat because of a missing ChatVerified header.

I found one bug:

Bob is using the old version (without this PR), Alice and Hocuri are on the new version (with this PR)

Bob creates a group with Hocuri (-> the group is protected for Bob)

Hocuri adds Alice, Alice sends a message into the group

-> Bob's device replaces the message by an error:

I think there are two possible solutions for the problem:

First make a release that accepts verifications via the _verified field, and that, ignores the verified property. And then make a release that actually removes protected chats.

From a user side, this is the most seamless.

Or: Modify this PR to still keep track of which group is regarded as 'protected' by group members running older clients. Then, continue to send the Chat-Verified: header in these groups.

From a user side, this is less seamless, because a user using the new version could still add a non-verified member, and users using the old versions will get an error when the non-verified member sends messages. OTOH, it arguably is already a UX bug that you can't add unverified members to some groups.

link2xt · 2025-08-16T02:36:25Z

test-data/message/verification-gossip-also-sent-to-from.eml

@@ -34,88 +34,112 @@ Content-Transfer-Encoding: 7bit

 -----BEGIN PGP MESSAGE-----

-wU4D5tq63hTeebASAQdATHbs7R5uRADpjsyAvrozHqQ/9nSrspwbLN6XJKuR3xcg


Payload here is replaced to have no Chat-Verified header inside and have _verified=1 in gossip headers. Payload got larger likely because rsop does not do compression and I had to reencrypt the payload.

iequidoo · 2025-08-31T07:38:05Z

src/receive_imf.rs

+            // is verified, but it is not known by whom.
+            context
+                .sql
+                .execute("UPDATE contacts SET verifier=?1 WHERE id=?1", (to_id,))


We mustn't reverify already verified contacts by "unknown contact" here, but i suggest to just merge at least the first commit from #7138, to make this PR a bit smaller and easier to review

Then #7138 needs to be not based on #7009. #7009 will need changes to sync message format after removing protected groups, so it's better merged after this change.

Hocuri

🎉 for 900 removed lines!

Partial review. Some smaller things, and I found a bug: #7116 (comment) (and on first sight, it's a rather bad bug, I think :/)

Concerning backwards-compatibility: I would prefer to deprecate rather than remove functions / change function signatures. This will mean that UIs can be compiled with current core until they update. (compiling with current core is useful for trying things out)

In a few weeks/months, we can then remove the deprecated functions.

(is_profile_verified and is_protection_broken can just be removed without any difficulties for UIs)

But I also see that it's multiple breaking changes at once, and that UIs anyways have to adapt before they can make a new release.

In any case, would be good if you write a checklist of what UIs have to do, and then copy-paste it into new issues in all UIs.

deltachat-ffi/deltachat.h

Hocuri · 2025-08-31T09:45:07Z

deltachat-ffi/deltachat.h

@@ -5278,8 +5236,7 @@ int             dc_contact_is_blocked        (const dc_contact_t* contact);
 *
 * @memberof dc_contact_t
 * @param contact The contact object.
- * @return 0: contact is not verified.
- *    2: SELF and contact have verified their fingerprints in both directions.
+ * @return 1=contact is verified, 0=contact is not verified.


Backwards-compat: At least Android expects this to return 2, rather than 1:

return dc_contact_is_verified(get_dc_contact(env, obj))==2;

I think it's easier to just keep returning 2 - at some point in the future, we will probably get rid of the C bindings, anyways.

Hocuri · 2025-08-31T12:54:29Z

deltachat-ffi/deltachat.h

- * @param chat The chat object.
- * @return 1=chat protected, 0=chat is not protected.
- */
-int             dc_chat_is_protected         (const dc_chat_t* chat);


Backwards-compat: Current UIs still check this because they are supposed to only let the user add verified contacts to protected chats.

So, would be nice to keep (and deprecate) this for another release, and let it always return 0.

Hocuri · 2025-08-31T13:14:49Z

src/receive_imf.rs

    mimeparser: &MimeMessage,
 ) -> Result<()> {
-    if mimeparser.get_header(HeaderDef::ChatVerified).is_none() {


Did you test how group members using the old&new version interact? E.g., would be bad if the old version rejects messages sent by new Delta Chat because of a missing ChatVerified header.

I found one bug:

Bob is using the old version (without this PR), Alice and Hocuri are on the new version (with this PR)

Bob creates a group with Hocuri (-> the group is protected for Bob)

Hocuri adds Alice, Alice sends a message into the group

-> Bob's device replaces the message by an error:

I think there are two possible solutions for the problem:

First make a release that accepts verifications via the _verified field, and that, ignores the verified property. And then make a release that actually removes protected chats.

From a user side, this is the most seamless.

Or: Modify this PR to still keep track of which group is regarded as 'protected' by group members running older clients. Then, continue to send the Chat-Verified: header in these groups.

From a user side, this is less seamless, because a user using the new version could still add a non-verified member, and users using the old versions will get an error when the non-verified member sends messages. OTOH, it arguably is already a UX bug that you can't add unverified members to some groups.

Hocuri · 2025-08-31T13:22:43Z

python/tests/test_1_online.py

    assert chat3.is_promoted()
    messages = chat3.get_messages()
-    assert len(messages) == 2
+    assert len(messages) == 3


Why is it 3 messages instead of 2 now? Was no "Messages are end-to-end encrypted" message added before?

Yes, previously unprotected groups had no such message.

Hocuri · 2025-08-31T13:23:21Z

python/tests/test_3_offline.py

-        assert len(sysmessages) == 3
+        assert len(sysmessages) == 4


Same here, why is it 4 messages now rather than 3?

Hocuri · 2025-08-31T13:24:34Z

src/aheader.rs

+    // `_verified` attribute is an extension to `Autocrypt-Gossip`
+    // header that is used to tell that the sender
+    // marked this key as verified.
+    pub is_verified: bool,


Would be nice to document this at https://github.com/chatmail/core/blob/main/spec.md

Hocuri · 2025-08-31T13:30:43Z

src/aheader.rs

+        if self.is_verified {
+            write!(fmt, " _verified=1;")?;
+        }


Might this make the first line of the Autocrypt: header too large, so that some email providers reject it?

I don't know why, but the first line of a random message in my DeltaChat inbox already is quite large:

Autocrypt: addr=******@*********.de; prefer-encrypt=mutual; keydata=********************************************************************** ****************************************************************************** ******************************************************************************

Apparently some line-wrapping isn't working here - anyways, the _verified=1; will make this line even longer, not sure if this will create any problems.

_verified is only sent in Autocrypt-Gossip which goes into encrypted part.

rustc-ice-2025-08-19T18_52_16-154475.txt

This mechanism replaces `Chat-Verified` header. New parameter `_verified=1` in `Autocrypt-Gossip` header marks that the sender has the gossiped key verified. Using `_verified=1` instead of `_verified` because it is less likely to cause troubles with existing Autocrypt header parsers. This is also how https://www.rfc-editor.org/rfc/rfc2045 defines parameter syntax.

Create unprotected group in test_create_protected_grp_multidev The test is renamed accordingly. SystemMessage::ChatE2ee is added in encrypted groups regardless of whether they are protected or not. Previously new encrypted unprotected groups had no message saying that messages are end-to-end encrypted at all.

link2xt force-pushed the link2xt/ykltkokxntvk branch from c7e6f1c to e180d7f Compare August 16, 2025 02:32

link2xt commented Aug 16, 2025

View reviewed changes

link2xt force-pushed the link2xt/ykltkokxntvk branch 4 times, most recently from 867c2ba to 780ca50 Compare August 17, 2025 02:31

link2xt mentioned this pull request Aug 17, 2025

Remove is_profile_verified() and is_protected() APIs #7112

Open

link2xt force-pushed the link2xt/ykltkokxntvk branch 3 times, most recently from 53d6a2d to a6301b9 Compare August 17, 2025 06:22

link2xt added the verified-chats label Aug 17, 2025

link2xt changed the title ~~feat: verify contacts via Autocrypt-Gossip~~ Verification v2 Aug 17, 2025

link2xt mentioned this pull request Aug 17, 2025

Expire the verification-property of QR codes #7111

Open

link2xt force-pushed the link2xt/ykltkokxntvk branch 11 times, most recently from 88700f4 to 5b31b5d Compare August 19, 2025 21:16

link2xt changed the title ~~Verification v2~~ Remove protected chats Aug 20, 2025

link2xt force-pushed the link2xt/ykltkokxntvk branch from 6b67844 to 6052d1a Compare August 20, 2025 03:33

Hocuri mentioned this pull request Aug 27, 2025

feat: Synchronize encrypted groups creation across devices (#7001) #7009

Open

link2xt force-pushed the link2xt/ykltkokxntvk branch from 6052d1a to 4e956bd Compare August 28, 2025 06:47

link2xt marked this pull request as ready for review August 28, 2025 08:37

link2xt force-pushed the link2xt/ykltkokxntvk branch from f1389dd to 701dafe Compare August 28, 2025 08:48

link2xt mentioned this pull request Aug 30, 2025

Expire authentication tokens #7140

Draft

iequidoo mentioned this pull request Aug 30, 2025

fix: Don't reverify contacts by SELF on receipt of a message from another device #7138

Merged

iequidoo reviewed Aug 31, 2025

View reviewed changes

Hocuri reviewed Aug 31, 2025

View reviewed changes

link2xt force-pushed the link2xt/ykltkokxntvk branch 4 times, most recently from 0b8c3ca to 904e5a3 Compare September 1, 2025 15:58

link2xt mentioned this pull request Sep 1, 2025

Various protected group changes #7146

Open

link2xt changed the base branch from main to link2xt/lypqsuxkxulr September 1, 2025 16:00

link2xt force-pushed the link2xt/lypqsuxkxulr branch from ec369bc to 4ab43d6 Compare September 1, 2025 16:16

link2xt force-pushed the link2xt/ykltkokxntvk branch from 904e5a3 to d8c237b Compare September 1, 2025 16:16

link2xt force-pushed the link2xt/lypqsuxkxulr branch from 4ab43d6 to 355ceaa Compare September 1, 2025 16:28

link2xt force-pushed the link2xt/ykltkokxntvk branch 2 times, most recently from 6d9ce24 to b91322a Compare September 1, 2025 16:56

link2xt added 2 commits September 1, 2025 20:42

api!: remove public APIs to check if the chat is protected

fc14cca

link2xt force-pushed the link2xt/lypqsuxkxulr branch from 2000568 to 04eda07 Compare September 1, 2025 20:48

link2xt force-pushed the link2xt/ykltkokxntvk branch from b91322a to 49f6b8d Compare September 1, 2025 20:48

link2xt changed the base branch from link2xt/lypqsuxkxulr to link2xt/recv-gossip-verified September 1, 2025 20:52

link2xt added 4 commits September 1, 2025 22:49

api!: remove Chat.is_protected()

7448869

refactor: remove ProtectionStatus

8e0c298

api!: return boolean value from dc_contact_is_verified()

83a9265

link2xt force-pushed the link2xt/ykltkokxntvk branch from 49f6b8d to 83a9265 Compare September 2, 2025 01:38

		@@ -34,88 +34,112 @@ Content-Transfer-Encoding: 7bit

		-----BEGIN PGP MESSAGE-----

		wU4D5tq63hTeebASAQdATHbs7R5uRADpjsyAvrozHqQ/9nSrspwbLN6XJKuR3xcg

Uh oh!

Remove protected chats #7116

Are you sure you want to change the base?

Remove protected chats #7116

Uh oh!

Conversation

link2xt commented Aug 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Hocuri left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

link2xt Aug 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

link2xt commented Aug 16, 2025 •

edited

Loading

Hocuri left a comment •

edited

Loading

link2xt Aug 31, 2025 •

edited

Loading