cancel
Showing results for 
Search instead for 
Did you mean: 

Suggestion - spam grading

N/A

Suggestion - spam grading

The PlusNet spam filter is OK, but it still allows me to download the spam so I have to filter it into another folder.

I'd ideally like to not have to download it but there is still a risk of false positives....

How about a grading system where we can choose what to do with an email depending on its SPAM grading e.g. obvious SPAM like Cheap Viagra would get high grade (e.g. SPAM9) but some of the more innocent appearing ones would get a lower grade (e.g. SPAM1)..

Then on the PlusNet settings you can choose what level of spam grading you wish PlusNet to kill (saving bandwidth...) and which level you'd rather download and sort yourself.
4 REPLIES
Plusnet Staff
Plusnet Staff
Posts: 12,169
Thanks: 18
Fixes: 1
Registered: 04-04-2007

Suggestion - spam grading

I know we were looking at options to do with spam to allow you to delete it, the spam system already uses a scoring system as part of the checks so its something that in theory could be done. You'll notice headers like this

X-SpamFiltered: By PlusNet (MXInternal v1.00)
X-DSPAM-Result: Innocent
X-DSPAM-Processed: Mon Mar 5 11:32:27 2007
X-DSPAM-Confidence: 0.7616
X-DSPAM-Improbability: 1 in 320 chance of being spam
X-DSPAM-Probability: 0.0000
X-DSPAM-Factors: 27,
FYI, 0.00607,
FYI, 0.00607,
CDR, 0.00621,
C+B, 0.00822,
C+B, 0.00822,
To*plus.net, 0.00861,
To*plus.net, 0.00861,
SDR, 0.00872,
SDR, 0.00872,
R+#, 0.01000,
R+#, 0.01000,
S+#, 0.01000,
S+#, 0.01000,
Q+S, 0.99000,
Q+S, 0.99000,
Subject*Username, 0.01000,
SHP, 0.01000,
SHP, 0.01000,
SHU, 0.01000,
SHU, 0.01000,
3DX, 0.01000,
3DX, 0.01000,
#(T, 0.01000,
#(T, 0.01000,
T(#, 0.01000,
T(#, 0.01000,
Received*(8.13.7, 0.99000

in some of the mails (some get marked as spam by one of the other spam systems we have so skip the dspam checker). Will see what I can find out.
N/A

Suggestion - spam grading

I'd love it too if Plusnet could blackhole all the spam, although it would be even better if you sould track down and close down the spammers at source.

I can't tell you how impressed I am at the way the new program "learned" how to sort spam. It took a couple of weeks, but for the last week or so it's been 100% accurate.
I now use message rules to send whatever is labelled [-SPAM-] into a spam folder. Norton never approached that level of accuracy.

Mind you I did keep sending the spam back labelled, and I suppose lots of other people did too.

Maybe ISPs and users together can beat the spammers.
Ianwild
Grafter
Posts: 3,835
Registered: 05-04-2007

Suggestion - spam grading

The source of most spam these days is individuals computers, so it's near impossible to stop it all at source.

There was an interesting piece on Radio 4 on the topic this evening. Should be available here:

http://www.bbc.co.uk/radio4/science/clickon/

Ian
carrot63
Grafter
Posts: 599
Registered: 12-07-2007

Suggestion - spam grading

I moved two of my domains away from PN a few months ago to hosting/mail at www.fuzioned.com , leaving one on PN hosting. It was around the time the PN spam platform threw a wobbly, and really I'd just had enough, although previously it was reasonably accurate in sorting good from bad.

I switched on the fuzioned spam checker without a great degree of confidence (I wasn't expecting miracles from five quid hosting), but have been very surprised at how accurate it is using a scoring system that places a string of "s" characters in the headers. There are also few false positives. Headers look like this:

Quote

From: "Lyman Myrtle Vega" <xxxxx@xxxxxx.com>
To: "Wendy" <xxx@xxxxx.nl>
Subject: Kristine wants you to check out this shop
Date: Wed, 07 Mar 2007 15:38:06 -0800
X-Priority: 3
X-FuZioned-MailScanner-Information: Please contact the ISP for more information
X-FuZioned-MailScanner: Not scanned: please contact your Internet E-Mail Service Provider for details
X-FuZioned-MailScanner-SpamCheck: spam, SpamAssassin (not cached,
score=24.42, required 5, autolearn=spam, BAYES_99 3.50,
DNS_FROM_RFC_ABUSE 0.20, DNS_FROM_RFC_POST 1.71,
DNS_FROM_RFC_WHOIS 1.45, HTML_MESSAGE 0.00,
RAZOR2_CF_RANGE_51_100 0.50, RAZOR2_CF_RANGE_E8_51_100 1.50,
RAZOR2_CHECK 0.50, RCVD_IN_BL_SPAMCOP_NET 1.56, SPF_FAIL 1.14,
URIBL_JP_SURBL 4.09, URIBL_SBL 1.64, URIBL_SC_SURBL 4.50,
URIBL_WS_SURBL 2.14)
X-FuZioned-MailScanner-SpamScore: ssssssssssssssssssssssss
X-FuZioned-MailScanner-From: xxxxx@xxxxxx.com


The string of "s" characters is easy for the filters in Eudora to read and grade the mail (except when my father wrote yesssssssss!!! after a rugby match), and above a score of about 8 you can really discount anything as junk and just chuck it without looking (there is one exception to this in my case).

Since early December, I've had about 20,000 mails in total, via PN and Fuzioned, with about 70 percent coming through the Fuzioned server. Running a search through my mail for this 3 months reveals:

Total of genuine mail wrongly marked as spam:
Plusnet ([-spam-] in subject) - 213
Fuzioned - 10, of which:
5 mails scored 7
1 mail scored 6
4 mails scored 5


I also picked out some sample spam from the last couple of days from both servers and checked them to see how well they had been scored (using 8 as the threshold for 'definite spam' in Fuzioned):

PlusNet (218 spam mails checked):
correctly identified as spam - 177
not marked as spam - 41

Fuzioned (217 spam mails checked):
correctly identified as spam (score 8 or higher) - 191
not marked as spam but suspect (scored 5-7) - 24
not marked as spam (scored 4 or less) - 4

The false positives on fuzioned were made worse (by 6) by one mailing list that insists on using a hotmail return address, html formatting and embedded gifs. PlusNet generated false positives for general mail, but also for my router logs, topic reply notifications from this forum and mails sent by my webcam using the PN servers.

As the Fuzioned server generates far fewer false positives, its a lot easier to deal with the mail, as you can at least be pretty sure the legit stuff gets through - although if I had a large number of contacts using Hotmail it might be worse. The marginal stuff is easy enough to quarantine for a double check.

The great thing about it is that you can set the threshold for blackholing junk entirely on the server without using specific usernames - useful as I use a different alias for wach site I register with, and it is a long, long list that is not really practical for an opt-in system (ex. plusforums@MyDomain.co.uk).

Whatever it is that Fuzioned use, Plusnet should get it in one form or another as the current systems accuracy and lack of gradation leaves much to be desired.