Spam platform problem

[#1]

I noticed this morning that I was getting an unusually large amount of spam, and sure enough, having looked at a few headers, I'm seeing lots of instances of 'X-Dspam-Improbability: 1 in 98689409 chance of being spam' - which has come up before when the spam-checking wasn't functioning on the mail server in question. It only seems to be happening on mxcores 01 and 18 - could someone look in to this please?
Thanks,
Thomas

[#2]

Hi Thomas,
Can you supply me with the headers from a few of these please? IIRC, the last time this happened alot of the X-DSPAM headers were defaulting to 0.4 (or is it 0.04 - I forget?) - Is this happening again?
Edit: We've been working on the mx.lasts this morning but sunmxcore18 is one of the mx.cores.

[#3]

Here's one from my mailbox:

Envelope-to: abc@xyz.plus.com
Delivery-date: Tue, 18 Sep 2007 10:55:44 +0000
Received:  by pih-sunmxcore18.plus.net with spam-scanned (PlusNet MXCore v2.00) id 1IXak3-00033r-If 
	  for abc@xyz.plus.com; Tue, 18 Sep 2007 10:55:44 +0000
X-Daemon-Classification: INNOCENT
Received: from 71-80-3-048.dhcp.sffl.va.charter.com ([71.80.3.48])
	  by pih-sunmxcore18.plus.net with esmtp (PlusNet MXCore v2.00) id 1IXajw-0002ip-FV 
	  for abc@xyz.plus.com; Tue, 18 Sep 2007 10:55:43 +0000
Received: from [71.80.3.48] by eforward4.name-services.com; Tue, 18 Sep 2007 05:56:27 -0500
Message-ID: <01c7f9e2$8fb97690$30035047@fernan>
From: "Will Mcrae" <fernan@plugvoip.com>
To: <abc@xyz.plus.com>
Subject: US $ 99.95 buy now Viagra 100mg x 30 pills
Date: Tue, 18 Sep 2007 05:56:27 -0500
MIME-Version: 1.0
Content-Type: multipart/related;
	type="multipart/alternative";
	boundary="----=_NextPart_000_0006_01C7F9E2.8FB97690"
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2800.1409
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1409
X-PN-VirusFiltered: by PlusNet MXCore (v4.00)
X-DSPAM-Result: Innocent
X-DSPAM-Processed: Tue Sep 18 11:55:43 2007
X-DSPAM-Confidence: 1.0000
X-DSPAM-Improbability: 1 in 98689409 chance of being spam
X-DSPAM-Probability: 0.0023
X-DSPAM-Factors: 27,
	vanishing+Dim, 0.40000,
	vanishing+Dim, 0.40000,
	X-PN-VirusFiltered*MXCore, 0.40000,
	Received*Sep, 0.40000,
	Received*Sep, 0.40000,
	Subject*Viagra, 0.40000,
	hspace=0+src="cid, 0.40000,
	Content-Type*charset="us, 0.40000,
	Content-Type*charset="us, 0.40000,
	equiv=Content+Type, 0.40000,
	ends?+From, 0.40000,
	ends?+From, 0.40000,
	face=Arial+size=2>XVI, 0.40000,
	watch, 0.40000,
	watch, 0.40000,
	endless, 0.40000,
	endless, 0.40000,
	Message-ID*<01c7f9e2$8fb97690$30035047+fernan>, 0.40000,
	ends?, 0.40000,
	ends?, 0.40000,
	Received*Tue, 0.40000,
	Received*Tue, 0.40000,
	Received*2007+10, 0.40000,
	What+is, 0.40000,
	What+is, 0.40000,
	from, 0.40000,
	from, 0.40000

[#4]

Yep, to save me editing out addresses etc., I'll just say that mine look exactly the same as John's, complete with all the 0.40000s. (Although I will post the headers if you do particularly need them.)

[#5]

Thanks guys, I've raised problem 46487.
Seems reminiscent of this.
Edit: Should now be fixed. It would appear that a problem with last night's training caused the Bayesian spam database on two of the 22 mail delivery servers to 'forget' what they had learnt with regards to spam email. They've been 're-educated' and should be behaving themselves now

[#6]

Well, I haven't seen anything slip through the net in the last couple of hours so it looks like you're right. Thanks for sorting it out quickly.

[#7]

Sunmxcore18's now doing exactly the same thing, so could it be looked in to please.

[#8]

I'll get this looked into, have you got any headers of this? PM them to me if you prefer.

[#9]

Ok, will do in a tic, though the problem's exactly the same as last time - improbability's 1 in 98689409, factors are 0.40000, etc..

[#10]

Cheers, got them. Will raise a problem asap.

[#11]

This should now be fixed (we've restarted Dspam).
Thanks for bringing it to our attention again Thomas.
We're working on a script that will run to periodically copy and clean the spam database before reinserting it and restarting Dspam. We'll try this once a week to begin with and see if the problem crops up again.

[#12]

Surely one could script a Nagios job to test for this on each relay?
Script an event that sends a mail through each particular relay to a specified mailbox, sleep for a short while, then check each mailbox, parse the headers for 0.40000 and respond with a pass/fail.
At least it would get the "flachenblinkenlights" going to alert someone
B.

[#13]

Thanks Bob (that was quick), and no problem.

[#14]

Quote from: Barry

Surely one could script a Nagios job to test for this on each relay? Script an event that sends a mail through each particular relay to a specified mailbox, sleep for a short while, then check each mailbox, parse the headers for 0.40000 and respond with a pass/fail. At least it would get the "flachenblinkenlights" going to alert someone B.

Damn, I'm having a quiet day at work today.
So, I found a nicely updated version of Mailping here
That does the mail delivery part.  Simply modifying the .pl script to check for the 0.4000000 headers would give you a lovely, handy Nagios script to tell you before your customers do
B.

[#15]

Thanks Barry, look interesting.
I'll flag it to the Net Ops guys. I know Si was working on a script that runs on DSpam and raises an alert if an unnatural volume of messages are accepted as clean but this may be a better way of doing things.