Plusnet
Saturday 21st November 2009Login | Register | Help
Pages: 1 ... 12 13 [14]

Dynamic IP Blocking

« Reply #208 on 26/09/2007, 13:30 »
Looking at a single mail delivery server last Tuesday, we can see that approximately 33,000 emails were actually /delivered/ to customers' mailboxes.

On Monday of this week, the same server was responsible for delivering around 23,000 emails (with the ACL in place).

Over a similar time period, and since the changes we made have been withdrawn, this server has delivered a staggering 37,000 emails Shocked

Given the fact that we have 22 mail delivery servers you can make the assumption that the total volume of email delivered is roughly 22 times the values above.

...............
With the ACL blocking = 506,000 emails
Without the ACL blocking = 814,000 emails


But how much will the traffic increase because users with their own mailservers are forced to use ISP servers because of dynamic blocking? I have a so called static IP from PN which is not static so at the moment have to use relay.plus.net for AOL and Hotmail. If PN think that is form of spam checking is going to increase then I will be using relay more and increase PN traffic.

As pointed out earlier in this thread one reason that I use my own server is because the lack of SMTP authentication so can't use PN outgoing server when away from home. When is authentication going to be introduced.

Also believe that a warning should be shown in the control panel when selecting a Static IP that in fact this is an permanent allocation of a dynamic IP.
Logged
« Reply #209 on 26/09/2007, 13:41 »
As pointed out earlier in this thread one reason that I use my own server is because the lack of SMTP authentication so can't use PN outgoing server when away from home. When is authentication going to be introduced.

Also believe that a warning should be shown in the control panel when selecting a Static IP that in fact this is an permanent allocation of a dynamic IP.

If you are running your own email server then why not set it up so you can use your email client through it wherever you are? I've got mine set up that way and I've also got squirrelmail set up so if I haven't got my laptop to hand I can use that. Takes a bit of configuration to make sure your server isn't an open relay but once its set up.....

Of course once you do that you realise that every single PC in the entire IP range owned by hinet is actually a zombie that thinks it will use you as an open relay (even if you aren't)
Logged
« Reply #210 on 26/09/2007, 17:04 »
But how much will the traffic increase because users with their own mailservers are forced to use ISP servers because of dynamic blocking? I have a so called static IP from PN which is not static so at the moment have to use relay.plus.net for AOL and Hotmail. If PN think that is form of spam checking is going to increase then I will be using relay more and increase PN traffic.

itsme, we're talking about emails sent to the mail delivery servers though, not the relays. Neither can I really see where you're coming from. If these customers weren't using their own mail servers then surely they'd be using relay anyway? Huh? Ok, there's the potential for more open relays etc. but as mentioned before we're pretty much on top of the housekeeping as far as the relays are concerned.

Quote
As pointed out earlier in this thread one reason that I use my own server is because the lack of SMTP authentication so can't use PN outgoing server when away from home. When is authentication going to be introduced.

Quite possibly sooner than you think! Wink

Quote
Also believe that a warning should be shown in the control panel when selecting a Static IP that in fact this is an permanent allocation of a dynamic IP.

I think that would cause more confusion than its worth.

Bob, I have an idea about the mail server platform that may be food for thought.
</snip>

Thanks Barry, I'll certainly try and find out what implications there would be if we tried implementing something like this.

« Last Edit: 26/09/2007, 17:06 by Bob »

Bob Pullen
Plusnet Comms Team
Service Status :: RSS :: Email

twitter / plusnet
Logged
« Reply #211 on 26/09/2007, 17:13 »
Thanks Barry, I'll certainly try and find out what implications there would be if we tried implementing something like this.

It was just off the top of my head, and I'm sure there are likely to be implementation problems.

However, I'd like PN to have the best performing email system around.  After all, it benefits me too Wink

B.
Barry Zubel : plusnet Community Site Forum Moderator
I'm a customer, not an employee
Logged
« Reply #212 on 27/09/2007, 17:36 »
I wonder if you could configure the load balancers to direct mail to specific relays based upon the ACL procedure you put into Exim and have subsequently rolled back.

After a bit of investigation, it turns out that this is certainly possible. It would involve investing quite heavily in additional hardware.

Quote
Consider two groups of relays:

1.  "Aggressive" relays, which provide a full DSPAM and CLAM filter
2.  "Passive" relays, which have a reduced DSPAM configuration

There isn't really a "more aggressive" Clam and Dspam setup available although that's not to say we couldn't look at other solutions. It's worth noting though that the blocking was introduced to reduce the load on the mxcores. Blocking in the ACL significantly reduces the impact on the servers. If we started processing and scanning all the messages then we wouldn't really gain anything from a hardware performance perspective (It's fundamentally the load and i/o on the delivery servers that creates a lot of the problems). You can only chuck so much hardware at the platform before you have to go after the root causes ie. spam.

Chucking hardware at the platform would be counter productive as it does little to tackle the root causes of the problem which is the ever increasing volumes of spam email.

Quote
When an inbound mail hits the load balancer, it performs a (fast) check on your dynamic IP ACL.  If it matches, then it passes the mail off onto the aggressive relays.  If it doesn't match, it is then sent through the more passive relays.

The thought behind it is that the majority of spam originates from dynamic IP addresses.  As long as you have the capacity to process this 'spam' email on the aggressive relays, that leaves the passive relays more time to process their (likely genuine) mail more expediently.

This would, however, put more load on the load balancers, if they can handle this sort of ACL check at all!

It's a bit like QoS for Email - those mails *fairly likely* to be spam could potentially be delayed a short while, whereas mail that is *fairly unlikely* to be spam will get processed by the faster 'passive' relays.

That way, all messages are processed, none are bounced with a 550, and you can start to identify methods on the aggressive relays with which to further fine-tune the process. 

You could even tag the mail passing through each relay "type" with an: X-Header identifying 'Passive' and 'Aggressive'.  Personally, I would be more than happy to join a trial whereby any mail that was misrouted could be sent to abuse@plus.net for further investigation.

This would help you to generate your 'whitelist', with the only customer impact being the *possible* delay of mail for a short while.

Again, resources are the concern here. Whilst I can clearly see the benefit of such an implementation from a customer perspective, it doesn't go any way toward preventing our mail delivery servers from exploding!

OK, so 'exploding's' probably a little bit colourful an eventuality, but I'm sure you get the picture! Wink
Bob Pullen
Plusnet Comms Team
Service Status :: RSS :: Email

twitter / plusnet
Logged
« Reply #213 on 27/09/2007, 18:01 »
Okay, so how about working it the other way around.

Inbound mail that matches the ACL just gets processed via relays that have *zero* DSPAM configuration, but are automatically marked as, for want of a better expression 'suspect'.

Mail that doesn't match the ACL gets processed in the usual way.

This would turn the tables on the processing time.    Stuff from dynamic IP's would require considerably less processing on the mail platform, the assumption being that it's already spam (or un-whitelisted mail servers).

This would allow you to start to create your own whitelist of remote mail servers, reduce the load on the mail platform quite considerably (?)  and allow you to work towards an approach where you do finally block mail with a 550 based on the ACL - after spending maybe a couple of months building up the whitelist so that the transition would be much less painful.

I appreciate that with a hardware investment required, that this isn't necessarily the ideal way forward.  It does have some positive options though:

1.  It reduces load on the mail relays, albeit at the expense of some load on the balancers
2.  It ensures that no legitimate mail is immediately blocked
3.  It compartmentalises the DSPAM processes onto fewer relays (which will be handling a much lighter load of mail)
3.  It reduces the requirement for running the DSPAM process on the relays that receive mail that matches the ACL
4.  It allows your customers to participate in building up the whitelist for the eventual (and inevitable) move to 550 rejection based on ACL
5.  No mail will be falsely rejected!

Again, much thought straight off the top of my head again Smiley

B.
Barry Zubel : plusnet Community Site Forum Moderator
I'm a customer, not an employee
Logged
Pages: 1 ... 12 13 [14]
Jump to:  

Related Sites

Community Apps

Here at Plusnet we're always trying to use clever open source things to make our lives easier. Sometimes we write our own and make other people's lives easier too!

View the Plusnet Open Source applications page

About Plusnet

We sell broadband, phone, VoIP and more to homes and businesses in the UK. Winner of 9 out of 11 Categories in the 2008 USwitch survey. Winner of "Best Consumer ISP" at 2008 ISPA awards. Voted number 1 in the Broadband Choices 2008 survey.

© Plusnet plc All Rights Reserved. E&OE

Powered by SMF | SMF © 2006-2008, Simple Machines LLC

Add to Technorati Favourites