Paul_B

I agree, that would certainly do it, at least I'd have control.

Email is ALWAYS the first point of contact for my new customers as I only sell on-line - I do not provide a phone number or 'surface' address on my web site.

Absolutely not! Which is one of the reasons we took the decision to roll back. We didn't appreciate what the scale of the impact would be. Neither did we expect to see the volumes of trusted email being rejected that we did. The vast majority of the examples I saw had been rejected as per design. We were also able to justify those that weren't (which was due to the way subnets were blocked - although we removed this late on Friday). No matter what we do (and we do need to do something), some customers *are* always going to be affected by the changes.

We need to ensure that we communicate any future implementation better, have clearly defined support content in place, and where possible progressively roll the changes out in a passive environment before any active blocking is done. It's worth bearing in mind that we had this system running on our secondary mail servers for days before this thread was started, which was a good indication that the impact on you guys wasn't going to be as bad as it ended up being!


Rethinking the strategy is exactly what we're doing. There are alternatives. We have (and still are) trialling numerous third party solutions (Postini, Critical Path & Ironport to name a few) - We have been working alongside these suppliers very closely indeed and partially outsourcing a solution to a third party is very much a possibility. Feedback on any of these suppliers is welcomed, as are any suggestions for alternatives?

I'm not sure I fully agree with the comment about our attitude towards customers. We listened to what our customers were saying in this instance instead of burying our heads in the sand, and whilst it could admittedly have been done sooner the ACL rejecting was withdrawn. We will be making every effort to engage the Community to a greater degree and ensure everybody is aware of the potential impact of any future implementation.

We're in the run-up to the Christmas period and spam is already starting to rise. It *will* get worse, and something needs to be agreed over the coming weeks.

We've already seen load shoot up since rolling back last weeks changes and left unaddressed there is absolutely no doubt whatsoever that our email platform will start to seriously suffer.

If someone  has a mail server that is on a "static" but "dynamic" IP address (e.g. any customer of Plusnet) then if they need to accept email then they need a valid MX record don't they?
I can't imagine most spammers create valid MX records so could that be used at all?

Lets face it - if you are running an email server that is exposed and sending/receiving directly and you don't have valid MX records then its your problem

Fair enough, that was really a response to Chris.  If you judge the sucess of the IP blocking measures put in place on a strictly technical basis . . . then I suppose he might be technically correct.

I read Chris' comment as suggesting that that the measures had worked as PlusNet had intended - which clearly they have not.  If I misunderstood then I apologise.

I do appreciate your efforts to find a solution and get involved in these discussions.

regards
Tom

Hi there,

Sorry if I wasn't clear before, you are correct in saying that the measures worked on a technical basis as they should. Obviously this had a bigger impact than anticipated.

So, if you knew that legitimate emails were going to be lost, why no communication of this to your customers before the measures where rolled out . . . and no option to opt out!???

We use IronPort at work and the only annoyance (which might be the way we have it set up) is that there seems to be no easy way of telling it that its missed something, but I've never had it trap something it shouldn't have done

No worries Tom, I understand where you're coming from.

Just to put things into perspective a little, here's some ball park stats:

Looking at a single mail delivery server last Tuesday, we can see that approximately 33,000 emails were actually /delivered/ to customers' mailboxes.

On Monday of this week, the same server was responsible for delivering around 23,000 emails (with the ACL in place).

Over a similar time period, and since the changes we made have been withdrawn, this server has delivered a staggering 37,000 emails

Given the fact that we have 22 mail delivery servers you can make the assumption that the total volume of email delivered is roughly 22 times the values above.

With the ACL blocking = 506,000 emails
Without the ACL blocking = 814,000 emails

So that's around 60% more email our mail platform has to manage without aggressively targeting suspected spam at the ACL level. Guaranteed a *very* small percentage of this email was from trusted sources (if you recall the figures I posted yesterday, we'd received around 500 contacts/requests for whitelisting as a result of the work). This is why we had to make a very difficult decision when we chose to roll the changes back. Even more to the point, it's why we need to arrive at an alternative or revised implementation sooner rather than later.

Doesn't that imply that your domain is hosted on PlusNet's servers. Lots of people have it externally with a different Domain Host, so the MX records would equally be external to PlusNet.

If PlusNet woke to their ridiculous domain pricing structure (£24 for 2 year .co.uk), more domains would probably be registered & hosted directly with PlusNet, which would have meant less emails were being redirected-in from domain hosts with dodgy server settings!

How about addressing that issue and having a special offer to transfer domains into your hosting platform, as that would solve a lot of these issues in one swoop!

I would never touch PlusNet for registering a domain currently (on price alone), although I do have one of my domains "hosted" with PlusNet. I have registered (and still Host) all my domains through easily.co.uk and they carried on redirecting to Force9 right through last week, so their servers are up to the job. They are £10.50 for 2 years for of .co.uk (so not the cheapest) but very flexible domain hosting & web/email forwarding without adverts is included, which as I say works seamlessly with PlusNet.

If PlusNet got their domain registering down to about the £8-£10 mark (for 2 years), I'm sure many would use them in the future. I know I would dump easily and move them all in to PlusNet for registering and hosting.

Mike

I'd go with PlusNet for their domain hosting, but after all they are an ISP, so they can't introduce lower pricing schemes because they simply aren't in the domain registration business, so can't compete with the "bigger players".

That's how I understand the situation, anyway.

It doesn't cost anything in most cases to transfer your domain in for hosting. It's only if you choose to use us as the registering agent that we charge (or if you've reached the hosting allocation on your account).

Linky!

It wasn't possible to offer an opt out as the changes were to the Exim configuration on the servers themselves and were applied across the board without an API to handle any sort of exclusion.

Bob,

I know I can register elsewhere, and transfer in for free (because I have done it!), but my point is that many DON'T know this.

I was talking about getting prices in-line with the market-place, then having a special offer for people who transfer registration & hosting in to you... something appealing to pull all those domains in... then the problem would be greatly reduced for next time you try this IP blocking situation.

I should have been clearer with my terminology.

It might even be worth doing a "marketing" drive on the ability to transfer domains in for free, and for them to be hosted for free on paying accounts. With a simple step-by-step guide on how to do it. In fact the guide probably already in your help&support section, so it probably just needs an awareness drive.

Mike

Thanks Bob.

I'm now subscribed to service updates by email and will try to find the time to read each one thoroughly.

The trouble is, the title reads 'Incomming email delays'.

Not - 'Major changes to email platform that may result in email being bounced'.

Bob, I have an idea about the mail server platform that may be food for thought.

I wonder if you could configure the load balancers to direct mail to specific relays based upon the ACL procedure you put into Exim and have subsequently rolled back.

Consider two groups of relays:

1.  "Aggressive" relays, which provide a full DSPAM and CLAM filter
2.  "Passive" relays, which have a reduced DSPAM configuration

When an inbound mail hits the load balancer, it performs a (fast) check on your dynamic IP ACL.  If it matches, then it passes the mail off onto the aggressive relays.  If it doesn't match, it is then sent through the more passive relays.

The thought behind it is that the majority of spam originates from dynamic IP addresses.  As long as you have the capacity to process this 'spam' email on the aggressive relays, that leaves the passive relays more time to process their (likely genuine) mail more expediently.

This would, however, put more load on the load balancers, if they can handle this sort of ACL check at all!

It's a bit like QoS for Email - those mails *fairly likely* to be spam could potentially be delayed a short while, whereas mail that is *fairly unlikely* to be spam will get processed by the faster 'passive' relays.

That way, all messages are processed, none are bounced with a 550, and you can start to identify methods on the aggressive relays with which to further fine-tune the process. 

You could even tag the mail passing through each relay "type" with an: X-Header identifying 'Passive' and 'Aggressive'.  Personally, I would be more than happy to join a trial whereby any mail that was misrouted could be sent to abuse@plus.net for further investigation.

This would help you to generate your 'whitelist', with the only customer impact being the *possible* delay of mail for a short while.

Thoughts?

B.

But there is already a GUI where I can say whether I want PlusNet to filter spam (or not). It must be easy to generate a whitelist for those willing to accept all email.

It has to be quicker to to do a file lookup than a DNS lookup.

Note that Exim is open source - you can amend the source to look at this whitelist and recompile exim.

Obviously this would require testing.
Can I suggest that any time a significant change is made that you ask for a contribution from some beta testers who are not financially reliant on email?