Have spam filters been trained over XMAS?

[#1]

The most negative thing to happen, for me anyhow, is PN has appeared to have stopped using the Spamcop blacklist. I say this because the spamcop header has been missing from all e-mails for the last 2 weeks or more.
So far, 100% of the undetected SPAM has been known to be through an open relay according to Spamcop.
For example see http://www.spamcop.net/sc?id=z1594309386z127336ac937383e53d6cf055bbd1f6e3z
SW.

[#2]

Quote from: Oldjim

Quote from: jelv At least they have (finally) changed things today to make DSPAM ignore the Postini headers. Not yet they haven't - just checked my spam folder - 3 of the 4 messages are forum replies Typical result Quote X-DSPAM-Factors: 15,     Envelope-to*james, 0.00020,     Subject*Topic, 0.00307,     X-Mailer*SMF, 0.00591,     To*james, 0.00735,     X-pn-pstn*2, 0.99000,     Delivery-date*2008, 0.99000,     net/forum/index, 0.01000,     Received*0005sz, 0.99000,     X-pstn-settings*1, 0.99000,     Received*([64.18.4.11]), 0.99000,     X-pn-pstn*Spam, 0.99000,     X-pstn-settings*0.1500), 0.99000,     X-pstn-settings*(0.1500, 0.99000,     Received*2008, 0.99000,     X-pstn-addresses*plus.net>, 0.01000

Yep, all of the headers to my spam tagged emails look like dspam has used the postini headers.
I've removed Spam Checking from all my accounts now.

[#3]

I first posted that DSPAM was being 'broken' by Postini headers on 13th December.
I posted about it several times in several related threads, but it is only now being picked up.
Tony

[#4]

In fairness, i've not noticed lots of false positives until today.  So it's only today that i've been affected.  I suspect the training done yesterday has caused the problem.  IIRC it's something that happens overnight once the mail has been manually vetted.
I'd be interested to hear how many mails it was trained with, if they were equal spam / not spam and if they were humanly checked.
But my guess is it's only people on Postini who have a problem (albeit a fairly major one!).  Lots of people who don't regularly check their spam folder could be completely oblivious to the issue.
If anything, those people not on postini may see more spam slip through the net due to dspam being more used to expecting postini headers for spam.  Who knows?

[#5]

I'm on the postini trial, and have just released a lot of legitimate e-mail, which had no problem getting through before haven't appeared in the inbox.
I checked the spam folder and they're all there, starting around 27th of December.
I've now turned off the spam filter.
My initial pleasure at the absence of spam once the postini thing kidded in, has been somewhat dented!
Everyone on the trial should check their spam folder!

[#6]

Quote from: Liam

In fairness, i've not noticed lots of false positives until today.

Originally, DSPAM was factoring Postini header entries as 'innocent' and so DSPAM was missing lots of spam. It now seems to be factoring them as 'spam' instead.

[#7]

Guys,
I was interested in another thread that was going on so hit 'Notify Me' link to keep up with replies. Guess where the notify e-mail went? Correct into my spam box and marked as spam.
Well you've gotta laugh now ain't you....

[#8]

Hang on, it turned up 20 mins later in my inbox as well, not marked as spam....
I'm confused.

[#9]

I'm confused too.
I just turned off spam protection at 'Manage My Mail' and now LOADS MORE spam is coming through marked as [SPAM].
H0ly link:censored ! how many will there be tomorrow?
I normally get 300 a day and I thought none was being stopped.
So it seems the spam filter is doing something. (which is a question I asked weeks ago).
Moderators note by James_H : Removed attempt to bypass swear filter. : Please read the link:rules

[#10]

I see a problem has been raised and this has been Service Statused 🙂
Cheers Bob.
http://usertools.plus.net/status/archive/1199352814.htm
http://www.plus.net/support/service/problems/problem.php?intProblemId=48952

[#11]

Quote from: Richard_261

I'm confused too. I just turned off spam protection at 'Manage My Mail' and now LOADS MORE spam is coming through marked as [SPAM]. H0ly link:censored ! how many will there be tomorrow? I normally get 300 a day and I thought none was being stopped. So it seems the spam filter is doing something. (which is a question I asked weeks ago). Moderators note by James_H : Removed attempt to bypass swear filter. : Please read the link:rules

I've just been back and switched it on again.
I had about 600 spams this morning, half of which were marked as [SPAM].
I haven't had a chance to scrutinise them yet to see if they're genuine or not.

[#12]

I'm on the Postini trial and I turned off DSPAM a few days ago.
I now deal with the resulting Spam locally.

[#13]

Quote from: pwebb

Quote from: jelv Having Postini on the horizon is no answer for people who have faced 80-90% spam as compared to real emails all over the holiday. You have dropped a spherical object on this - big time! I'm not sure why Postini is relevant to this conversation as we have not done anything different with our spam systems than we usually would?

Phil,
I get the impression that you rather took exception to my comments. In view of what has happened in the last few days perhaps you would like to reflect and consider whether it would be appropriate admit "You have dropped a spherical object on this - big time!" and to email an apology to all users for what has happened over the last three weeks.
If anything, the complaints in these forums are louder than ever. We now seeing people regretting referring friends because of the amount of spam they are receiving, moving their email elsewhere etc.
A couple of extracts from the EOD blogs:
[quote=End of Day: 2nd January 2008]THE TOP 5 CALL REASONS ARE:
  1. SPAM - This seems to be the most popular lately, unfortunately this time of year is terrible for spam so our postini trial has come at a good time.
[quote=EOD: 4th January]The CSC also let us know that the Top 5 issues people have been calling about today are:
    * Spam
I've also noted a comment in the EOD about Darren doing the MAC key analysis - I'd really like to know what percentage of users are citing problems with spam as their reason for leaving at the moment. Somehow I suspect it would be far to embarrassing to reveal (and it's the end of January one where I suspect this will peak).

[#14]

Quote from: jelv

Quote from: pwebb Quote from: jelv Having Postini on the horizon is no answer for people who have faced 80-90% spam as compared to real emails all over the holiday. You have dropped a spherical object on this - big time! I'm not sure why Postini is relevant to this conversation as we have not done anything different with our spam systems than we usually would? Phil, I get the impression that you rather took exception to my comments. In view of what has happened in the last few days perhaps you would like to reflect and consider whether it would be appropriate admit "You have dropped a spherical object on this - big time!" and to email an apology to all users for what has happened over the last three weeks. If anything, the complaints in these forums are louder than ever. We now seeing people regretting referring friends because of the amount of spam they are receiving, moving their email elsewhere etc.

John,
I didn't take exception to you comments, but was answering what you asked and pointing out that the reason for the lack of training was nothing to do with the imminent rollout of Postini.
I've read the thread that I think you are referring to and most of the people posting in it with issues are not on the Postini trial.
As you will have seen from the problems raised and Service Status posts, we've been training dspam a lot more than usual over the past number of days to make sure that we remove and reference to Postini being used in the factors (which will not be in the headers for these customers) and get it back to being as effective as it can be.
The fact that these users are receiving huge amounts of spam is therefore not down to a lack of training of dspam which is what my original reply to you was about but either dspam being less effective, us not getting the right training messages for it, the spam changing so quickly, back scatter or many other reasons.
The poor performance of dspam is the reason that we went out looking for a commercial offering to provide a better level of service for our customers.
Phil

[#15]

Phil,
DSPAM detects on a word level, not on a part-word level.
Spammers have realised that if they run permutations of words together into long words, DSPAM does not see any consistency in the words to be able to detect them as spam - however much you train it.
I have done courses in 'expert systems' in the 1980's during and I understand both how they operate, and how they train.
Unless DSPAM maintenance allows you to create additional rules such as "word length greater than 25 characters" then you are on to a loser with this 'long word' spam.
Tony