Send Later incompatible with PGP encryption (Error: encryption)

[farfallosa]

Since the actual update, "Send later" shows an "error: encrypted" in drafts folder. I use OpenPG and in TB 78x OpgenPG is integrated and does automatically encrypt drafts - so e-mails set for send later, too.
Therefore "send later" does not work. :(

[jperryhouts]

When Enigmail was an independent plugin there was a way to turn off the "encrypt draft messages" option, but it seems that option is gone now that PGP is integrated into Thunderbird. There's really nothing I can do about that, Sorry.

You might consider opening a bug with Thunderbird if you want this fixed.

PS. This was always incompatible with Send Later, but the latest update is more proactive with telling you about it. Nothing changed in the latest update to make this incompatible.

[fabacab]

@farfallosa Did you follow through with @jperryhouts's suggestion to escalate this to the Mozilla Thunderbird dev team? I'm looking through their Bugzilla bug tracker now but can't see a relevant bug and want to ensure I'm not spamming them. The closest existing ticket I see that may be relevant is their Bug ID 1168907, "Saved draft email is always encrypted when encryption is set to off for the composing email," but this is 6 years old.

[fabacab]

Meanwhile, @jperryhouts, would you be willing to share with us a little more about why encrypted drafts are incompatible with Send Later? At first blush, it seems to me like an encrypted draft should not be a showstopper for Send Later functionality, so long as the email being sent "later" does not require private key material (e.g., it is not being signed).

What is the technical block preventing Send Later from working with encrypted accounts? I ask because I have some familiarity with both JavaScript and RFC 4880, as well as a modicum of extension development experience, I'd like to see this functionality implemented, and I can potentially devote a modest number of hours to helping implement a solution. I'd like to help, if I can.

[jperryhouts]

This wouldn't be a problem if Thunderbird encrypted Draft messages using the recipient's public key, but the point of encrypting draft messages is that you can open them up again and continue editing them. So instead Thunderbird encrypts those messages with your own public key. That makes them both unreadable to your recipient, and also blocks Send Later from opening them without user interaction (note there might be some way for send later to open them if you don't have a passphrase on your private key, but that's not really a solution).

Two things will need to happen before this really has a workable solution. One is that Thunderbird needs to allow some programmatic access to the raw content of those messages, presumably just by allowing you to store them unencrypted in Drafts. The other is that Send Later needs to then be able to encrypt their contents before placing the message into the Outbox, when its scheduled time arrives. This will likely also require some accommodation on Thunderbird's side, since I don't know how easy it is for extensions to interact with the new builtin PGP system.

So far I haven't put much thought into the latter problem, because it's blocked until the first one is solved. But if you're interested in working with the Thunderbird devs on this I'd be happy to stay in the loop to ensure that whatever solution does get implemented in Thunderbird is aligned with Send Later's use case.

[farfallosa]

@fabacab thanks for nudging me. I just posted a new bug report at bugzilla (https://bugzilla.mozilla.org/show_bug.cgi?id=1681168)
It would be great if you could help fixing this problem. Because I don't think that TB will change their approach within a reasonable period of time, although this would be the best solution. (Not in Bugzilla but in forums this bug was reported several times.)
Note : The draft could be permanent decrypted by clicking into the subject line. But this does not have any effects on the date-time-stamp in the drafts column of a send later saved mail. @jperryhouts

[jperryhouts]

Note : The draft could be permanent decrypted by clicking into the subject line.

Not sure I understand what you mean by this. When you view a draft message it's of course decrypted for you to see its contents, but the message as stored on disk / server is still encrypted. Unless I'm misunderstanding your comment.

[fabacab]

@jperryhouts:

Thunderbird encrypts those [draft] messages with your own public key. That makes them both unreadable to your recipient, and also blocks Send Later from opening them without user interaction (note there might be some way for send later to open them if you don't have a passphrase on your private key, but that's not really a solution).

Hmm, I often send signed email via Thunderbird's built-in OpenPGP support and am not asked for my private key's passphrase. As I understand it, either the key or its passphrase is stored in memory (along with access to my other account passwords) after I provide my Thunderbird "primary password" after the program first starts up. I'll probably have to dig around in the codebase to see what's exactly happening, but this experience is why I'm surprised you'd need to prompt the user for their OpenPGP private key passphrase at all.

In any event, thanks for your work thus far and if I end up being able to contribute in some way I'll do so here.

@farfallosa:

I don't think that TB will change their approach within a reasonable period of time

The Thunderbird team is actively working on OpenPGP support. Every recent release since built-in OpenPGP support has included changes and bugfixes to the way OpenPGP preferences are handled, so now's a good time to poke them about this sort of thing.

Also, depending on the complexity of the issue, we might not need them to do more than they've already done. For example, I just now noticed that there is a autoEncryptDrafts boolean exposed via Thunderbird's Config Editor. Toggling it doesn't seem to have an immediate effect, but maybe in combination with some other existing config option, we can find a working solution or at least a workaround to help unblock Jonathan.

[jperryhouts]

I assume that your private key is stored in memory, because the whole thing feels pretty transparent when you're using it, but I don't know that can be counted on to always be true. For example, plain gpg via the command line usually stores your private key in memory for some time period, then asks you to enter the passphrase again once that time period has elapsed.

There's also the problem that I'm not sure how to interact with the new PGP system via the extension API. The extension API is very lacking right now, and to my knowledge nothing exists to expose the encryption system to add-ons, even indirectly.

@fabacab Whatever investigating & work you're interested in taking on regarding this is very welcome. Let me know what you find.

[jperryhouts]

I'm going to re-open this issue because I'm positive this is not going to be the last I hear from users about it, and it will be useful to have somewhere to point them.

[farfallosa]

@jperryhouts As far as I can say, once decrypted, a draft remains decrypted. I.e. even after a restart of TB the draft remains decrypted and does not need to be decrypted again. Therefore I assume that the draft is saved plain after a first decryption.

Note: To mention, this happens even if the draft is neither signed nor encrypted.

[jperryhouts]

Use Ctrl+Shift+U to look at the message source, and then tell me that message is not encrypted...

[farfallosa]

It is … I wonder how TB manages to keep a decrypted view over restarts even though the message itself remains encrypted.

[dpesch]

I've found a simple workaround for TB78 with OpenPGP:

If you set the [Security | Encryption Technology setting] in a new mail to S/MIME, the email is saved unencrypted (Thunderbird V. 78.6.1) and could be processed by Send Later. However, you have to change this setting every time you open the draft again.

Of course, this only helps if you want to send an unencrypted e-mail and, like me, have not configured S/MIME.

[farfallosa]

@dpesch Thanks for the workaround. But that would mean that I can't send OpenPGP signed or encrypted mails. Therefore this is rather out of the question for me.
It would be fine if one could come to a solution that already existed with Enigmail, namely to be able to deactivate the encryption in the draft

[jph-sendlater]

@farfallosa Of course this isn't a solution, but it's miles ahead of the current situation, where even having PGP configured prevents Send Later from working at all. This workaround enables an important use case for people who need PGP only for communicating with certain contacts. I'd venture to guess that the majority of PGP users only encrypt a small fraction of their outgoing mail.

[dpesch]

@farfallosa you can decide for every new mail if you want to send it later (= non encrypted) or send it immediately with OpenPGP encryption. I fit exactly into jph's guess and only encrypt individual, particularly important, mails. Most of my recipients do not use PGP.

I use Send Later similarly: I only send a certain (= few) of my emails with it.

The workaround is not great, but better than nothing. I would also prefer it if the Thunderbird authors gave us the choice of whether we want to encrypt the drafts or not.

[farfallosa]

This could work: If Send Later would store emails not in draft folder, but eg in outgoing mail, the mail would not be encrypted and therefore Send Later could work, independent of the TB OpenPGP settings.

[jph-sendlater]

Unfortunately, it's not that simple. Thunderbird has established tools for moving a message from its composition window to either Drafts, Outbox, or immediate send (all of which pass through PGP first, if it's configured). It doesn't have any such mechanism for placing raw messages in any arbitrary folder.

[maxmichels]

Is there any progress by TB? I have multiple accounts configured but PGP only for one account. It is frustrating that these emails are not send by SendLater but i have to remember each time so send it manually...

[jph-sendlater]

It looks like the Thunderbird side of this issue will be resolved in the next ESR version! https://bugzilla.mozilla.org/show_bug.cgi?id=1681168
It will be possible to disable encryption on local drafts (and thus on scheduled messages), but that probably means Send Later messages will have to also be sent unencrypted.

Now the ball is in my court to enable encrypting scheduled messages.

[1337-TIM]

great new !!! :)

[1337-TIM]

Hi , me again ...
I just downloaded the thunderbird V 91.1.0 and still not working with encryption ... I saw there's a V 9 of senditlater on the way (my actual is the last official) : Do you think it will work with your 9.x new version ?
It's so useful to use send it later , I hope it will works soon

Thank you for the job youre making 🥇

[jph-sendlater]

As an aside, Send Later version 9.0 is a big overhaul of a lot of Send Later's internals, but it won't have many user-facing changes. I plan to release that via ATN soon. That isn't really relevant here, but just FYI.

[1337-TIM]

thank you for the information ! I'm gonna change my preferences ; waiting for the solution, that will allow me to schedule un encryped mails and keep the encryption for the others - not so bad
thank you again

[stefan11]

Hi, I tire this but this seems to make it impossible to use encryption at all. That does not work for me. Send later seems to be a great tool for work place hygiene (not spamming each other out of working time hours) but killing encryption for this seems to be too high a price to pay. I would be really glad if there would be a way to fix this. =:-)

[spi43984]

Hi, any news or timeline on using encryption with send later? Would be much appreciated...

[jph-sendlater]

@spi43984 No news that I'm aware of.

[YannisDelmas]

Some news: I use TB v 91.8.1 (64 bits) and found the following solution. May be you should consider implementing it in this project or suggest it in the message to the user (poke @jph-sendlater #427).

In TB advanced configuration, adjust the following parameters (I founded these to be true in my configuration):

• mail.identity.id*.autoEncryptDrafts → You may want to suppress these.
• mail.identity.default.autoEncryptDrafts → You may want to set it to false.

[m342]

In released 91.9.0 version
this "seems" to be available via account settings per account.
https://support.mozilla.org/en-US/kb/new-thunderbird-91#w_openpgp-new-advanced-settings

[mach-o]

Yes, this worked beautifully for me in v 91.8.1 by simply going to Account settings >> End-to-end Encryption >> then disable "Store draft messages in encrypted format" under Advanced settings

I actually have TB configured so that my drafts folder is under a different account (Local Folders) than the actual account (an IMAP account) and although the setting is not available for local folders, changing it for the IMAP account worked.

I'm very pleased because I just now encountered this problem for the first time, and immediately came upon this solution that seems to only have been around for a handful of days or weeks.

[spi43984]

Yes, this worked beautifully for me in v 91.8.1 by simply going to Account settings >> End-to-end Encryption >> then disable "Store draft messages in encrypted format" under Advanced settings

Send Later sends originally encrypted emails delayed but does not encrypt the emails anymore, so this is unexpected for most users as they encrypt their outgoing email but Send Later sends them out later unencrypted.

[jikamens]

As of release 10.0.9 of Send Later and any newer release, it'll prevent you from scheduling a message if you have encryption or signing or drafts encryption enabled. This is the best I can do at this point; the APIs for me to actually be able to sign or encrypt scheduled messages simply don't exist and I don't know if they ever will. I'm closing this discussion because there's nothing more I can do about it until the APIs are better.

[kaie]

@jikamens Which API do you currently use for sending emails?

If I understand correctly you want to send emails in the background, without having an email composer window open. You want to remember whether an email had encryption and signing turned on, and you probably also must remember which encryption technology was selected, and probably some other flags, e.g. whether subject encryption was on or off.

If I understand correctly, you'd need a modified version of the API that you are currently using, which allows you to specify the additional message security flags?
(CC @jobisoft )

[jikamens]

@kaie I don't use an API, I copy raw messages into Outbox and then tell Thunderbird to send them from there.

[kaie]

I see.

I think what you need is a way to completely prepare the message, exactly as it would be sent out, and then queue that copy for later use, right?

I think this functionality already exists and could help you.
In the composer window, we have the command "file / send later".
If you use that, Thunderbird will apply signing and encryption for all recipients, and will send it in the Outbox.

IIUC, all that you want on top of that is scheduled sending at the desired time, right?

[kaie]

Well, I realize, you don't want the signature creation date to be at the time of message preparation, but at the time of message sending.

[kaie]

You'd need an API that loads the draft into a (hidden) composer window, which would restore all the message security flags as saved in the draft's state, and then tell that composer window to send out the message.

[jikamens]

Yes, a hidden compose window would probably be the quickest way to get to a working API, since the message composition functionality is currently so tightly intertwined with the compose window that it would be difficult to separate it out.

[jikamens]

Putting it another way, if the TB team did decide to separate the presentation of the compose window from the message formatting and sending functionality, they wouldn't want all that code to be in two places, so they would then have to basically rewrite the entire compose window to leverage the newly separated functionality. That's probably too heavy of a lift. It also seems higher risk than the hidden-compose-window trick you mentioned.

[jikamens]

Hey @kaie. If you don't mind, I'd like to ask, who are you and what has prompted you to weigh in on this ticket? I'm asking this because I have been working on this add-on and on Thunderbird for more than a decade, and I'm extremely familiar with what facilities are available within the application, and yet you seem to be asking me basic questions and making basic suggestions that are not adding anything to my knowledge.

I'm happy to get help from people who have expertise and knowledge I don't have who might be able to help me make progress in this area, but it is not helpful for me to have to spend time explaining things I already know and responding to suggestions I've already ruled out because they won't work.

So where exactly are you coming from here?

[jikamens]

I'm also confused about your thread of questioning here because I'm under the impression that the Thunderbird team has already started to work, on at least a conceptual level, at implementing an API to allow add-ons to leverage Thunderbird's mail-sending functionality without having to open a compose window. That's what I need, and the Thunderbird team already knows it's a need that add-on maintainers have, so it feels kind of weird that you seem to be trying to reinvent the wheel here and drive the discussion to a point of concluding something we already know.

[kaie]

Hi Johnathan, I'm a core Thunderbird developer, I'm the primary maintainer of the email encryption function, and I was the person who integrated Enigmail into Thunderbird a few years ago. I was made aware of this issue on Mastodon, and I'm trying to explore ways to potentially help you.

[jikamens]

Thank you, I appreciate that. As I noted in a separate comment, the only viable and appropriate solution I see to the problem outlined here is a proper add-on API for preparing and sending messages without going through the compose window. I don't see any reason why I or the Thunderbird team would spend time or effort building anything else.

[jikamens]

To respond to your earlier question, I can't just use the Send Later functionality to put the message into the Outbox at the time the message is scheduled both because of the problem you observed, that the dates on the encrypted/signed message will be wrong, and because once a message is in the Outbox my add-on has no control over when it gets sent. Even leaving aside the timestamp issue, it certainly would not be safe to tell Thunderbird to copy the message into the Outbox and then grab it back out of the Outbox, hoping that in the interim something else doesn't do a Send Unsent Messages command and cause the message to be sent at the wrong time. I'm not going to build something that introduces a dangerous race condition like that into my code.
And in any case that would be a gross kludge. The right solution here is to expose the mail-sending functionality in a supported add-on API.

[kaie]

Thanks Johnathan for allowing me to brainstorm here and for giving me your thoughts. It helped me to better understand the situation and the requirements. I cannot promise when we might be able to help with this, but I'm adding a note to our TODO list. I also don't know if @jobisoft already has plans for this functionality, I haven't yet talked to him - I probably should have, prior to reaching out to you.