cancel
Showing results for 
Search instead for 
Did you mean: 

PlusNet Gets Tough On SPAM

Community Gaffer
Community Gaffer
Posts: 12,807
Thanks: 636
Fixes: 62
Registered: 04-04-2007

PlusNet Gets Tough On SPAM

Evening all,

As most of you will undoubtedly be aware, this week has seen a few problems with the email platform. This post like a few others that I've made recently, is intended to provide a little more information on our continued commitment to ensuring that we are doing everything possible in order to provide a first class email platform that is resilient, scalable and most importantly reliable.

Background

Some of you will already be aware that we have recently installed a brand new mail storage platform and today started moving customers across to the new system. Further details regarding this work can be seen here. If that's a little too much to take in then make sure you check out the Service Status announcement here as the work will involve a small interruption to your service.

In addition to this, we've also completely rebuilt our mxcore mail delivery platform and almost doubled the number of servers responsible for handling mail collection.

The above changes should certainly be welcomed and will go a long way towards safeguarding the future reliability of the email platform. There is however one core contributor to the problems we have seen that this work doesn't address. That problem lies firmly at the feet of everyone's worst enemy... Spam!

Spam!

Spam is an issue on the rise. The amount of spam flying around the Internet is increasing; 2006 was the worst year for it so far; and this trend is expected to continue. It's clear just how significant a problem this presents by looking at the information gathered from our own systems. The following represents a snapshot of the volumes of email we were processing in the run up to Christmas:



We can clearly see that the volume of mail being processed by the platform increased by about 55% in the 2 weeks before Christmas alone!!!

Such increases in spam puts massive strain on the mail platform...

  • We need extra resources to process the mail as it arrives. Without enough processing power mails begin to queue which results in delays for customers.
  • Email that's awaiting delivery or awaiting collection has to be stored somewhere. This is what the storage platform is for. With increases like those exhibited above you can expect this to start running out pretty quickly unless you're effectively policing spam.
  • All those GB's of mail have to get to us somehow so of course we have to scale the transit connections to the rest of the Internet to cope with the additional traffic.


So what are we doing about this? Well, to begin with we're changing the way in which customers mailboxes are set up when they create an account. By default we have always provided a 'catch-all' email address. What this means is that email is delivered to you irrespective as to what comes before the '@' sign. This makes you very susceptible to spam as emails are often sent to random aliases. This is something that we see crop up all too often in these forums so it will be nice to see the back of it. We're looking to make these changes in the 3rd week of April and I posted a bit of a teaser back in January containing more information that can be seen here.

In light of the most recent spate of mail timeouts we've also taken the decision to make some configuration changes to the way in which our mail delivery servers handle spam. Before explaining these changes it's helpful to have an understanding of how the delivery servers handle your email.

Mail Delivery

When you send an email, the email program or web application you are using sends your message to a local mail delivery server (SMTP Server). The SMTP server then forwards the mail onto the server responsible for collecting and delivering messages for the person you're sending the email to. For emails sent to our customers this is our mxcore platform.

Now consider what would happen if our mxcore servers were unavailable or out of service. It's important that the email isn't lost so it makes sense to have a backup server. These servers are what form our mxlast platform.

Legitimate email rarely needs to pass through our mxlast platform as it is very uncommon for the mxcore servers to be unavailable.

You wouldn't have thought this looking at the volume of email that passes through the server though:



The reason for this is that most of this email is spam. One trick that spammers often use is to directly target the secondary mail server of a domain (in our case the mxlasts). The reason they do this is because these servers are far less likely to have as stringent spam scanning as the primary server meaning the spammers email is more likely to get successfully delivered.

Our SPAM filtering platform

We have already added spam filtering to the mxlast platform as part of our recent mail delivery server upgrade. Whilst this helped to increase spam detection rates, it didn't help reduce the previously mentioned processing and storage problems. This is because the servers still have to deliver the SPAM to the customer.

Our SPAM filtering is a 'multi-layered' process. This basically means that we use a number of different SPAM filtering mechanisms before emails are delivered to the customer. If an email isn't identified as spam then it will be checked by the next process which will pass it on to the next process etc. If any of these processes identify the email as SPAM then the tag [-SPAM-] is appended to the subject line of the email and delivered to the customer.

Proposed Changes

On April 12th we will be making some changes to the way that our mxlast platform handles the mail that is sent to it. Details regarding this work have been posted as a Planned Maintenance Announcement here.

Each email will be checked against one of our SPAM filtering processes and if determined to be SPAM will not be accepted by the platform. This will greatly reduce the volume of email our servers have to process as there will no longer be the overhead associated with delivering these junk emails to customers.

Some customers have aired concern in the past about legitimate email they have received in the past that has been marked incorrectly as [-SPAM-]. It is worth mentioning at this point that this does *not* mean that we will now be dropping this email without the customer knowing.

These mistakes are the result of our Bayesian filter which is a completely separate process to the one we will be using to refuse email on the mxlasts. Because we use this process now as part of our spam filter we have been able to make sure we are confident as possible that no email will be refused incorrectly.

What does this mean?

Well most noticeably we hope it means that anyone plagued by large volumes of spam should see a marked reduction in the amount of junk email they receive.

We have estimated that these changes will reduce mail volumes on the mxlast platform by around 60-70%. That in itself is about 20-30% of the total volume of email passing through our platform. Whilst making SPAM less of a nuisance for customers we are also helping protect the integrity of the mail platform and reduce the potential for problems similar to those witnessed earlier this week.

We are aware that we have many customers who prefer to manage their own email by running their own SMTP server. We would like to assure these customers that the changes to the mxlast platform will not affect them and they will continue to manage their own email as normal.

Feel free to discuss these plans and ask any questions you may have and I'll be more than happy to try and source the answers.

Kind Rgds,

Bob Pullen
Plusnet Products Team
If I've been helpful then please give thanks ⤵

31 REPLIES
cirenna57
Newbie
Posts: 4
Registered: 31-07-2007

Tough on Spam

Bob
I am reading your message referring to Spam and when I reach the end of the following piece (The amount of spam flying around the Internet is increasing; 2006 was the worst year for it so far; and this trend is expected to continue. It's clear just how significant a problem this presents by looking at the information gathered from our own systems. The following represents a snapshot of the volumes of email we were processing in the run up to ChristmasSmiley I then have a space, I assume that there should be a graphic here, can you tell me why I cannot see anything?
I realise that this is probably something really obvious but please allow for the 73 years wear on my little grey cells.

Eric
chymorva
Grafter
Posts: 275
Registered: 29-08-2007

PlusNet Gets Tough On SPAM

Hi Eric,

Welcome to the forums.

Is it just that one graphic that you can't see ?
Near the top of the page is says 'Discussion Forums', at the RH end of the orange line below that is another graphic - can you see that one ?

Can you see the picture beneath my username (chymorva) at the LH side of this post ?
cirenna57
Newbie
Posts: 4
Registered: 31-07-2007

PlusNet Gets Tough On SPAM

Thanks for your speedy reply Phil, answer 1: yes I can see a silhouette of 3 figures.
answer 2: no there is nothing after your username.

Eric
N/A

PlusNet Gets Tough On SPAM

Are you able to see graphics on other sites? Try news.bbc.co.uk and plus.net.

What browser are you using to view the forums? If you are using Firefox check that you haven't blocked images from the domain "*.plus.net" or similar (in Firefox go to Tools -> Options -> Content and click the "Exceptions" button next to "Load images automatically", obviously making sure that it is ticked).

Are you using any internet accelerators?

What security/anti-virus software are you using?
Liam
Grafter
Posts: 2,083
Registered: 04-04-2007

PlusNet Gets Tough On SPAM

For reference, the image that you should be able to see is here :

http://www.binarybob.plus.com/table.PNG

Can you see it if you visit that link? Can you see the mail queue graph later on in the post?
Community Veteran
Posts: 3,364
Thanks: 15
Registered: 06-04-2007

Re: PlusNet Gets Tough On SPAM

Quote

Each email will be checked against one of our SPAM filtering processes and if determined to be SPAM will not be accepted by the platform. This will greatly reduce the volume of email our servers have to process as there will no longer be the overhead associated with delivering these junk emails to customers.


I am assuming that given I have totally turned off PN spam filtering by the portal. This change will not affect me?

Quote
Some customers have aired concern in the past about legitimate email they have received in the past that has been marked incorrectly as [-SPAM-]. It is worth mentioning at this point that this does *not* mean that we will now be dropping this email without the customer knowing.


This is the exact reason I have turned off the PN spam filter as I am happy with accuracy of MacOS Mail and JunkMatcher.

SW.
--
3Mb FTTC
https://portal.plus.net/my.html?action=data_transfer_speed
Community Gaffer
Community Gaffer
Posts: 12,807
Thanks: 636
Fixes: 62
Registered: 04-04-2007

Re: PlusNet Gets Tough On SPAM

Quote
Quote

Each email will be checked against one of our SPAM filtering processes and if determined to be SPAM will not be accepted by the platform. This will greatly reduce the volume of email our servers have to process as there will no longer be the overhead associated with delivering these junk emails to customers.


I am assuming that given I have totally turned off PN spam filtering by the portal. This change will not affect me?


Yes it will.

Quote
Quote
Some customers have aired concern in the past about legitimate email they have received in the past that has been marked incorrectly as [-SPAM-]. It is worth mentioning at this point that this does *not* mean that we will now be dropping this email without the customer knowing.


This is the exact reason I have turned off the PN spam filter as I am happy with accuracy of MacOS Mail and JunkMatcher.

SW.


When did you turn it off? I find that it's pretty accurate nowadays.

Rgds,

Bob Pullen
Plusnet Products Team
If I've been helpful then please give thanks ⤵

Community Veteran
Posts: 3,364
Thanks: 15
Registered: 06-04-2007

Re: PlusNet Gets Tough On SPAM

Quote
It is worth mentioning at this point that this does *not* mean that we will now be dropping this email without the customer knowing.


So how will I be told what messages have been dropped?

How do I recover them if I decide it was not a SPAM message?

SW.
--
3Mb FTTC
https://portal.plus.net/my.html?action=data_transfer_speed
Community Veteran
Posts: 3,364
Thanks: 15
Registered: 06-04-2007

Re: PlusNet Gets Tough On SPAM

Quote
Quote
This change will not affect me?


Yes it will.


So the Spam Filtering setting on the portal will be meaningless when this change is implemented?

SW.
--
3Mb FTTC
https://portal.plus.net/my.html?action=data_transfer_speed
cirenna57
Newbie
Posts: 4
Registered: 31-07-2007

PlusNet Gets Tough On SPAM

Lestones:
Have tried both the sites you showed and yes I do see graphics on them both.
I do use Firefox and all the ticks are there I have not had to touch anything.
What I have just found out is that my notebook shows the graphics, apart from the OS being XP Pro on this compared with XP Home SP2 everthing in Firefox setup is the same.
No internet accelerators as far as I know are on the system.
eTrust Antivirus
Comodo Firewall Pro
Adaware
Spybot
Spyware Blaster
Spyware Guard
Perhaps a little paranoid but hey it has paid so far.

Liam:
Your link showed me the details, as far as the mail queue graph farther down the post it is a no graphic situation, please be aware of my statement above, I can see everything on the laptop.
May I make an apology to one and all as this seems to be detracting from the original post of Bob's, being a newbie to posting on a forum I just jumped in to the post as this was where the fault was noticed, perhaps someone might put me right gently please.
Community Gaffer
Community Gaffer
Posts: 12,807
Thanks: 636
Fixes: 62
Registered: 04-04-2007

Re: PlusNet Gets Tough On SPAM

Quote
Quote
It is worth mentioning at this point that this does *not* mean that we will now be dropping this email without the customer knowing.


So how will I be told what messages have been dropped?


You won't. The above quote needs to be read in context.

Quote
How do I recover them if I decide it was not a SPAM message?


This won't happen. Hardly any legit emails get sent to the mxlasts and we're using clam spam at the mxlast level anyway. It's dSpam that's responsible for the false positives you sometimes see so this isn't going to present a problem.

Kind Rgds,

Bob Pullen
Plusnet Products Team
If I've been helpful then please give thanks ⤵

Community Gaffer
Community Gaffer
Posts: 12,807
Thanks: 636
Fixes: 62
Registered: 04-04-2007

Re: PlusNet Gets Tough On SPAM

Quote

So the Spam Filtering setting on the portal will be meaningless when this change is implemented?


Not at all. It will still mark messages as spam that are sent through the primary mail delivery servers (mxcores).

I think you might be misunderstanding my post. We are *not* dropping all email that would normally have been tagged as spam.

Kind Rgds,

Bob Pullen
Plusnet Products Team
If I've been helpful then please give thanks ⤵

Community Veteran
Posts: 3,364
Thanks: 15
Registered: 06-04-2007

Re: PlusNet Gets Tough On SPAM

Quote
The above quote needs to be read in context.



I thought I had. So it means I not understood the contents of this post at all. Sad

SW.
--
3Mb FTTC
https://portal.plus.net/my.html?action=data_transfer_speed
Plusnet Staff
Plusnet Staff
Posts: 12,169
Thanks: 18
Fixes: 1
Registered: 04-04-2007

PlusNet Gets Tough On SPAM

The way to look at it is that there are two sets of mail servers, mxcore and mxlast. mxcore is the main platform, mxlast the backup. Under normal circumstances the majority of legit mail goes to mxcore and a fair chunk of spam goes to mxlast.

The spam filter which you set on the portal will scan all mail from mxcore and mxlast with both Clam and Despam and mark the subject with [-SPAM-] if it thinks mail is spam.

If you don't have the spam filtering on then this mail doesn't get scanned.

The change we are making means that all mail from mxlast will be scanned by Clam and rather than marking it spam will reject it, but won't reject mail that Despam marks as spam.