Help! Emails I'm trying to send to Gmail accounts are bouncing!

[#1]

That SMTP diagnostic code looks disconcerting and looks like it’s yet another problem with Gmail…

https://support.google.com/mail/thread/173900861/message-could-not-be-delivered-due-to-missing-valid...

[#2]

This https://github.com/mattermost/mattermost-server/issues/20604 might shed a little more light...

[#3]

So, the key question here, is what bit of the end-to-end service is responsible for generating the Message ID?  Client or server?

I just sent a message to my Gmail account from my Outlook 2016 client and the Message ID is clearly there...

Subject: Test to see what happens
Date: Sun, 2 Oct 2022 13:55:48 +0100
Message-ID: <008901d8d65e$4d265fa0$e7731ee0$@mydomain.me.uk>

Mydomain is hosted by Plusnet and has a SPF record.

[#4]

---

@Townman wrote:

So, the key question here, is what bit of the end-to-end service is responsible for generating the Message ID?  Client or server?

---

I think it's Plusnet, rather than Gmail: Gmail is rejecting the messages because they don't have the right headers. I also suspect that it's not all of Plusnet's mail servers that have a configuration issue that's causing the headers to be incorrect - and that's why some messages are delivered successfully, but others fail.

I've got an open ticket with Plusnet on this at the moment, but I'm considering moving my domain away from Plusnet to an actual domain host. I have been with Plusnet for 17 years and when I signed up, the domain, email and webhosting were included. Since then, none of these features is included with new accounts, and they are becoming so legacy that I don't think Plusnet really wants to maintain them as services. I understand that decision from a business perspective, so I'm thinkint the best solution to give me robust service, might be to move my domain hosting away from Plusnet.

[#5]

Not sure who generates the Message-ID but according to RFC 2822

   Though optional, every message SHOULD have a "Message-ID:" field.
   Furthermore, reply messages SHOULD have "In-Reply-To:" and
   "References:" fields as appropriate, as described below.

   The "Message-ID:" field contains a single unique message identifier.
   The "References:" and "In-Reply-To:" field each contain one or more
   unique message identifiers, optionally separated by CFWS.

So strictly speaking it's not mandatory...

[#6]

So strictly speaking Gmail is in the wrong. So, [yet again] no new news here then.

[#7]

---

@garywood84 wrote:

I think it's Plusnet, rather than Gmail: Gmail is rejecting the messages because they don't have the right headers.

---

There is no expectation of Gmail generating the message-ID - it is a matter of it being the sending email client or it being the sending MX server.  Outlook with Plusnet's mail server clearly generates the Message-ID header.  I have never seen an email with that header missing, but I always use Outlook.  There are numerous references to this issue and they all cite Gmail as being the common recipient.

Further it is Gmail deciding to reject email messages contrary to the RFC rules - the header is OPTIONAL - therefore as we have seen so many times, Gmail is out of order here.  I have already referred to an article mentioning that this issue has been with Gmail support since earlier this year ... and remains un-responded to.

Separately it would be interesting to know where the header is generated and why in some circumstances it appears to be missing.

@bobpullen - do you have any thoughts here please?

[#8]

https://en.wikipedia.org/wiki/Message-ID says

Message-IDs, if present, are generated by the client program sending the email^[3] or by the first mail server.^[4] A common method of generating such ID is by combining the time and domain name, for example: 950124.162336@example.com.^[5]

[#9]

Hi Mister W

I'm using @barnsdall.co.xx in the emails, but the account config is using my @freenetname.co.uk with the appropriate server configs, checked and double checked. And, yes, i've tried it with my@freenetname address.

I've had this freenetname account for donkeys years, I think from around 1998/9, certainly before I moved house in late 2000.

Its certainly starting to affect our business as it was just the occaisional gmail bounce back, now its every single one. Even my son who regularly emails had our email reported as spam - so perhaps its gmail not recognising plusnet/freenetname etc and blacklisting them.

As per another poster has put, it may be time to move on from Plusnet.

[#10]

I'm using @barnsdall.co.xx in the emails

if you do that then then you have an increased likelihood of your emails being treated as spam or rejected.

You will not have an SPF record for the domain. If you want to use your domain for email , then you really need to get a hosted email package.

[#11]

---

@MisterW wrote:

https://en.wikipedia.org/wiki/Message-ID says

Message-IDs, if present, are generated by the client program sending the email^[3] or by the first mail server.^[4] A common method of generating such ID is by combining the time and domain name, for example: 950124.162336@example.com.^[5]

---

So, Plusnet could fix this by adding the header...

[#12]

Why only now, after all this time?

There has to be a logical reason for gmail now doing this, or don't trust that sort of email?

I've looked into SPF and am not able to have one as I don't host the domain name.

Anyway, as I said it may be time to move on.

[#13]

I've looked into SPF and am not able to have one as I don't host the domain name

and that's the problem!

SPF is being used more & more by mail systems to detect spam. Whilst a lack of Spf won't in itself cause email to be rejected , it will increase spam scoring.. In the past you could send out domain email via any mail server without problems, nowadays you need a properly hosted domain that provides for SPF and ideally DKIM 

[#14]

Thanks to @Hobo and @JasaMan for their time and the detail they've provided, I think we've managed to confirm what's happening. One of my colleagues in NetOps has explained to me, as alluded above, Google look to be aggressively marking the emails as spam, unless there's what's known as a valid MessageID in the headers.

Sending mail to Gmail addresses via webmail works as we'd add this MessageID, but some email clients don't seem to be when sending from a Plusnet email address. Unfortunately due to the nature of the issue, there's little we can do other than suggest you use webmail or a different email client that adds the required MessageID.

It may also be worth raising this with Google: https://support.google.com/mail/ as others seem to have done. 

[#15]

@Gandalf 

Another great conclusion.  It has helped to identify that the FROM address is not the Plusnet service email address which is being used to relay the emails.  That will give rise to complications if there is no SPF record for the sending address.

Given the comments on RFC specifications...

1. The inclusion of the header is optional, thus Gmail ought not to be mandating that it must be present - as suggested several times, another Gmail fail!
2. Given that the format of the message header embeds the SENDING email address, then that is something known to the sending client, not the server thus is the responsibility of the client - it might be worth users taking up that matter with their email client vendors

Thank you for grinding this one to a clear conclusion!