TB with IMAP sometimes deletes e-mails when moving to local folders

[#1]

I use Thunderbird with IMAP on Plusnet for my incoming e-mails. This generally works well, but after dealing with mail I like to move it to local folders on my own computer for permanent storage, so I have a full and proper record I can back up should I have any subsequent query.

Unfortunately, somewhere between TB and Plusnet's IMAP implementation there is a nasty bug which can result in e-mails being permanently lost. This is unacceptable, since a record of correspondence needs to be complete in the event of a dispute.

When the bug occurs, messages dragged from the inbox to the local folder do not move, and cannot be made to move. When TB is closed and reopened, the messages will be deleted from the inbox, and the same number of empty messages will be created in the local folder. They are not stored on the server and appear to be permanently lost.

I previously used POP, which didn't have this problem, but read messages were not being deleted from the server (despite TB being configured to do so), causing speed and bandwidth problems and the number of read messages built up over the years.

Does anyone else experience this problem with PN's IMAP?

[#2]

To me "moving" implies removal from the original folder after copying to the new location. However that should happen only after a successful copy. With IMAP deleting normally sets a deleted flag but the messages remain in place until the folder is compressed ("expunged").. Thunderbird wouldn't show them but webmail usually does in light grey text if it hasn't done the deletion itself. Webmail can "undelete" them, did you take a look?

I'm wondering if what you are experiencing is Thunderbird dropping the connection for reasons yet to be understood. For example what is mentioned before and after post https://community.plus.net/t5/Everything-else/Possible-packet-loss-issue/m-p/1518299#M18809. Theoriginator of that topic is using cloud services so the issue might not be associated only with Plusnet IMAP.

[#3]

Going into webmail was the first thing I did when I saw the messages gone from TB and replaced with blank ones like this:

As you can see, all that remains of the original four messages is the Forwarded flag for the first one. There was no sign of the messages on webmail, either in the inbox or the deleted folder. The time shown is the time of the attempted move, I think (or possibly when I closed and restarted TB to try to free the messages again).

TB does sometimes drop the connection when sending, but usually clicking Retry a couple of times gets the message sent. This is different; the drag or menu move operation just doesn't happen, and then the messages get lost when TB starts the next time. They flashed up briefly when TB restarted but then disappeared.

[#4]

This all reads as purely a TB failure. As David states a move should be a successful copy followed by mark for deletion. There seems to be a fair few recent reports of issues using TB which are not being experienced inconjunction with other email clients. You could mitigate the risk of this failure by you performing a copy to local folders and then delete from the server when you are sure they have copied successfully. Alternatively you could use a more reliable email client!

[#5]

@kjpetrie Do you have a date and time for when this happened so we can see if there is anything untoward in our logs?  

Thunderbird really shouldn't send a delete message till it's been told an email has been successfully moved..?!

[#6]

I found this while Googling.  It's a 10 year (!!) old bug in Thunderbird which sounds similar. 😮

https://bugzilla.mozilla.org/show_bug.cgi?id=462156

Unfortunately, Thunderbird deletes the source emails (on IMAP), even though it has not successfully copied them to their destination on a local folder. Instead, on the local folder, I get a long list of empty emails (no subject, no body, sometimes with reply and forward markers preserved), all with the same time stamp. The original emails are lost. They are not present in the email file. Rebuilding the index does not recover the data.

[#7]

Yes, that's it! I think the answer is obvious. TB is not a reliable IMAP client and I must migrate to another.

Except that after looking for a replacement - TB is the only major supported e-mail client for my platform. The others look like one-person amateur efforts which could disappear when their maintainers get bored/grow up/have children.

Still, at least I know it's not PN's fault.

[#8]

@Kelly, 17:11 on 8/3/18. It would be good to know if the logs show anything.

[#9]

@kjpetrie The bug report linked by @Kelly makes a disappointing read (for those with the stamina to do so). 157 comments with the useful contribution you added. The previous one was 7 months ago, and still no sign of acknowledging the bug, never mind fixing it.

Despite this I think Thunderbird remains usable with some inconvenience. Copying rather than moving emails to local folders should be used. Then after confirming successful copying the originals can be deleted.