relay.plus.net - slow to disconnect

[#1]

I'm working on my own SMTP server at the moment as part of a website project (incoming emails can be set to a web script etc hence the need for a custom SMTP server).
I've got several accounts that I've been sending test emails through and they're all ok except for two: Gmail and Plusnet.
Both take a while to disconnect - Gmail takes 30 seconds and Plusnet takes 1 minute. This is the same for both servers every time they connect and deliver email to my SMTP server.
I'm puzzled. To the best of my knowledge my server is thread safe, it's doing everything in a clean way and objects are created / released as and when needed however each new connection from PN takes one minute to disconnect. Even when the same connection delivers more than one email it still takes one minute to disconnect - not two or three.
Anyone from PN able to see anything in the logs that might help me?

[#2]

We cannot unfortunately.
Just to clarify what you are saying. Are are sending an email to our relay (ignoring Gmail for the moment, given lack of direct control of this), and it is taking 60 seconds for the transaction to close the connection when the QUIT command is issued?
Your messages could also suggest that this gap is also the time in which our relay forwards it to your server, or that the delay could be on another command other than quit.
I'm just trying to asses which command is actually being slow.
Are you able to provide a provide a packet capture (please provide this on a support ticket, if you are using SMTP authentication and not as a link on the forum), showing from before the connection opening, until after the connection is closed?
This will show the full flow and details of the SMTP transaction, showing specifically what parts are slow.
As an additional item, can you confirm you have ruled out any local firewall and/or intrusion detection that may be delaying data in any way.
I have been unable to replicate what is being described, but this isn't to say it is not happening. My suspicion is something like slow DNS queries, so a specific part of the SMPT transaction.

[#3]

I read this as him having his own SMTP server for incoming mails (which get forwarded to a web server) and it's the PN servers delivering the emails to him that are hanging on to the connection.

[#4]

Re-reading it, that is a possibility.
To be honest, a packet capture is still an ideal scenario if at all possible, as whilst we are obviously looking at a transaction in the opposite direction, it should help us see what is happening.

[#5]

Phil,
Having read this I might be able to help. We run our own mail server ( Mercury ) in the office and just looking at our logs for yesterday and I'm not sure if that also shows something strange on a connection from the PlusNet relay. I'm not an expert around email and SMTP but just looking at the logs there appears to be no QUIT from the PlusNet relay.
Is that normal ?
I can PM you the Mercury SMTP logs for some typical transactions including one from PlusNet if it will help. I obviously don't want to post them here without a lot of redaction and I'm not sure f that would render them useless...

[#6]

Quote from: jelv

I read this as him having his own SMTP server for incoming mails (which get forwarded to a web server) and it's the PN servers delivering the emails to him that are hanging on to the connection.

Exactly correct. Give that man a job!
My server receives no QUIT command from the PN relay server - after one minute the quit command is sent and the PN relay server disconnects from my server. It's exactly 60 seconds every time.
I could code around this, disconnecting the incoming connection in my onreceive event but I'd rather it be clean and tidy code wise only running the OnDisconnect event when the incoming connection voluntarily disconnects.

[#7]

Thanks,
I've raised a query to those in the know, to see what we can find out.

[#8]

Thanks Phil. It may be something I've missed and your server connection to mine could just be a timeout but if it is, it would be nice to know - then I'll know for definite what I'm dealing with rather than guessing.
Seems odd how the other accounts send to my server and quit instantly but gmail and plusnet don't.

[#9]

Hi Sprite,
Is your SMTP server sending a 250 completion/acceptance response following the <CR><LF>.<CR><LF> DATA termination? If not, that might explain why Plusnet and Gmail are sitting around (waiting for one) before timing out at 60 seconds and QUITing.
That aside, what's the problem with them sitting around for 60 seconds before leaving?
Mathew

[#10]

Quote from: MJN

Is your SMTP server sending a 250 completion/acceptance response following the <CR><LF>.<CR><LF> DATA termination?

Just checked with Putty, Yes it is.

Quote from: MJN

If not, that might explain why Plusnet and Gmail are sitting around (waiting for one) before timing out at 60 seconds and QUITing.

While it is sending a 250 code, it does indeed show symptoms of gmail and plusnet using a timeout which expires and sends a quit command. It's an odd one..

Quote from: MJN

That aside, what's the problem with them sitting around for 60 seconds before leaving?

Not really a problem but I'd like to know exactly whats going on in case its at error in my program that I need to fix. If I'm going to leave it running 24/7 as a windows service, it needs to be reliable right?

[#11]

As long as you're doing your bit, and sending the 250 OK acceptance, there's little more you can do. The ball is back in the senders court and short of disconnecting the session (which you shouldn't do) you'll just have to let it wait for the QUIT command to be sent.
Mathew

[#12]

I've just had a thought..
Is this some form of bandwidth saving technique employed by the big ISPs to save DNS lookups and initial connection bandwidth? - I've noticed that when the connection stays open it will still deliver new email many seconds after the last one via that same connection.
Is that whats going on? - Bandwidth / resource saving? If so thats fine, doesn't worry me, I just want to be sure my server isn't at fault.

[#13]

DNS results would be cached anyway in accordance with your TTL settings so there'd be no benefit there, and whilst some SMTP servers will reuse existing connections I would be surprised if they waited a whole minute for the chance to do so - that long a time period could easily offset any potential savings of not going through the connection/handshaking process as required.
The default Postfix timeout (smtp_connection_cache_time_limit) for allowing the opportunity to send another message is only 2 seconds which may not sound much but for high volume traffic that is more than enough time to decide whether it is worth hanging on to the open connection[1].
Mathew
Edit: [1] Actually, looking at the docs it seems that Postfix shares its open connection amongst any SMTP process whereas other MTAs (Sendmail and Exim are explicitly listed) may operate a non-sharing connection cache thus for them reuse is limited to within any given process. For this reason the timeout would have to be longer to achieve the same benefit, but 60 seconds? It seems rather anti-social. I'm sure Plusnet can verify based on their configuration (as you have summised, the same 60 second wait every time strongly suggests it is entirely intentional).

[#14]

My original thoughts had been connection re-use provided a queuable item within that mailer becomes available. However, I could not find any reference docs to confirm that.
Still awaiting a response on this though for you.
It makes sense to reuse an existing connection. Afterall, the resource is available on the local and remote side, though the biggest saving is a reduced number of checks that a remote MTA may need to perform. At the end of the day, if it does become a resource issue to have it open so long, you are likely to have more scalability issues than just that.

[#15]

Hi
Sorry, I've not got around to capturing the packets so can't supply that yet - I'm not really too bothered by it to be honest I just wanted to be sure my server wasn't at fault as the other accounts I've been using to test it have disconnected instantly.
From my POV its certainly looking like the resource saving may be the issue here because I can send another email through PN 30-40 seconds later and its still delivered via that same live connection. If the connection had hung then that wouldn't be the case would it (either my SMTP server would refuse to process the email or your server wouldn't deliver it). It seems logical that this is intentional and I think I'm pretty happy with that understanding of it now - especially since GMail are doing the same thing too. It's certainly not a resource issue for me to worry about if its open so long, I just wondered what was going on. I feel happy now that I can trust it 
Many thanks for your input!