Postini Email Security Trial
Quote from: Strat We cannot determine Postini's performance as we can't count blocked emails because we don't know about them.
Exactly ... and that's the single biggest downside of the system IMHO. There is little if any independent and 100% reliable proof one way or another as to just how well (or otherwise) the system is performing. Yes, you might be seeing less spam but exactly how much less and how much not-quite-spam is being removed at the same time is completely uncertain. Marketing 'statistics' or indeed SLA's (because they amount to the same thing in this particular instance) don't mean as much as they imply simply because there is no real and reliable means of substantiating them or proving beyond doubt that they are (or are not as the case may be) being achieved. One particular user (or group of users) may well feel that the success rate is <SLA but will be unable to prove it and therefore unable to claim whatever the SLA offers if the relevant figure is not being achieved. The only thing I can say is that in my case, there is no shortage of historic hard evidence of spam performance various so it will be pretty obvious just how well (or otherwise) the system is performing if/when I am effectively forced into using it. It will be a very interesting time to say the least
BTW, the reason I particularly queried your gut feeling of Postini being 'better' than Criticalpath Mr.Jelv is because of some rough figures you posted somewhere. I can't remember where I saw them TBH but I think you said something along the lines of "I would normally expect around 200 spams a day but have only received 20 today" and that is, in principle and wholly without being hard evidence, somewhat less than the alleged SLA. It's also worth noting that the SLA is perhaps (and almost certainly has to be I think) a VERY average figure across a very wide userbase over a long(ish) period of time. Whilst some users will no doubt be seeing near 100% performance, I suspect that others will be seeing very much less than SLA performance. The SLA can without doubt be shown to be being achieved with some users receiving 100% of the spam directed at them whilst others are receiving no mail at all because 100% of their genuine mail as well as the spam is being rejected.
Totally ignoring the good reputation of Postini (as I have absolutely no way of determining for myself if it is justified or not) I believe that the only real way to interpret the SLA's being bandied around is in exactly the same way as the 'upto' should be interpreted when considering a broadband connection offering performance of upto 8MB ! I personally have little faith in "We have been using Postini internally for ages and it's top notch, extra good, way better than anything else and reliable" stylee comments without any hard evidence to substantiate them because, as I'm sure you'd agree, we've heard similar comments before and such comments have turned out to be less than reasonable in reality for reasons various. That's not to say that 'problems' can't be fixed if they do materialise of course, just that it's a bit difficult to 'fix' any problems there might be if you can't prove that a problem does indeed exist in the first place
Edited To Add: Maybe PN would like to extend the SLA's to me personally and offer me a free month for each and every time I can demonstrate that the stated performance is unlikely to be being achieved
Surely, if the quoted SLA figures are so good and so readily achievable then this shouldn't be a particular problem for them should it 
1. There is a world of difference between marketing 'statistics' and SLA's. With an SLA there will be penalties if the performance figures are not met and therefore there I attach a high degree of credibility to these whereas I attach no credibility to marketing claims.
2. Before Postini the volumes of spam I received tracked the reports I see on various forums about spam volumes. This week there have been many reports that spam volumes are up, Plusnet have said volumes are exhibiting the same rise that they normally see in the approach to Christmas and one user's very useful graph that you might just recognise shows spam volumes for the last three weeks at the highest level since spam day:
(Click the graph for an explanation of the various coloured lines)
I'm seeing the lowest spam volumes I've seen for many, many weeks - in fact I would go as far as to say that the volume of spams getting through is probably back to before Spam Day (I suspect the last time I saw spam volumes so low the Plusnet mail system was totally dead so I was getting none!).
3. I am not aware of a single email that I was expecting that has not arrived - certainly all the newsletters etc. that I subscribe to have arrived.
2. Before Postini the volumes of spam I received tracked the reports I see on various forums about spam volumes. This week there have been many reports that spam volumes are up, Plusnet have said volumes are exhibiting the same rise that they normally see in the approach to Christmas and one user's very useful graph that you might just recognise
shows spam volumes for the last three weeks at the highest level since spam day:(Click the graph for an explanation of the various coloured lines)
I'm seeing the lowest spam volumes I've seen for many, many weeks - in fact I would go as far as to say that the volume of spams getting through is probably back to before Spam Day (I suspect the last time I saw spam volumes so low the Plusnet mail system was totally dead so I was getting none!).
3. I am not aware of a single email that I was expecting that has not arrived - certainly all the newsletters etc. that I subscribe to have arrived.
Quote from: MrToast
PN have stated an SLA for system availability. I've not seen mention of one for accuracy.
Quote from: pwebb
The Postini solution for the trialists has dropped around 70% on the edge which is definite spam (their marketing suggests around 50% on average). We have not got figures yet for how much of the total they are marking, but the SLAs we have is 98% detection rate with 0.0003% false positive rate.
(click the Quote from to see the full post)
Quote from: jelv I'm seeing the lowest spam volumes I've seen for many, many weeks - in fact I would go as far as to say that the volume of spams getting through is probably back to before Spam Day (I suspect the last time I saw spam volumes so low the Plusnet mail system was totally dead so I was getting none!).
I'm quite sure you are and I'm not in any way suggesting otherwise but you get what I'm saying don't you - it's very easy to 'have a good feeling about something and believe it to be right' but subsequently find out it's not quite as 'right' as you thought. I mean, how many times have you had to correct me because my thoughts/beliefs/memories are not quite as good as I thought they were at the time I
As far as spam volumes are concerned, I'm effectively seeing a 100% performance on my A/Cs using my own rather trivial system for last week - that's 100% 'rejection' of spam, 0% false positives and 0% false negatives. If I was doing nothing at all and Criticalpath had been rolled out and configured as PN had initially intended, I (and probably most other users) would be seeing ~96% rejected spam, ~4% false negatives and ~0% false positives. Not figures to be complained about by any means and all fully supported by hard evidence. Had this been the case then I'm quite certain that just about all users would also have been putting out the flags this weekend and cheering PN for a job VERY well done.
BTW, what a d@mn fine looking graph that is ! I really do think PN should be providing whoever that particular user might be with a free connection for services rendered
Whilst I would have been cheering, I'm not so sure I would have been cheering that loud.
DSPAM has given me a couple of false positives this week. Secondly with Criticalpath tagging and DSPAM but without Postini I would still be getting a high volume of tagged spam to be checked for false positives. With Postini the volume of tagged spam to be checked is much, much less and so I'm less likely to miss them.
I'm getting very, very close to turning off the Plusnet filter and relying on the Thunderbird junk mail controls. This has become more likely because Postini is getting rid of the image spam which was needing frequent training in Thunderbird.
DSPAM has given me a couple of false positives this week. Secondly with Criticalpath tagging and DSPAM but without Postini I would still be getting a high volume of tagged spam to be checked for false positives. With Postini the volume of tagged spam to be checked is much, much less and so I'm less likely to miss them.
I'm getting very, very close to turning off the Plusnet filter and relying on the Thunderbird junk mail controls. This has become more likely because Postini is getting rid of the image spam which was needing frequent training in Thunderbird.
Quote from: jelv DSPAM has given me a couple of false positives this week. Secondly with Criticalpath tagging and DSPAM but without Postini I would still be getting a high volume of tagged spam to be checked for false positives. With Postini the volume of tagged spam to be checked is much, much less and so I'm less likely to miss them.
Whilst it's perhaps totally unreasonable for me to comment since I have intentionally never used DSPAM or any other 'optional' filtering, going by the various comments made by a large(ish) number of users who have, I think it would be entirely reasonable to suggest that it's pretty useless in reality. Yes it helps up to a point I guess but the short-comings far outway any positive contribution from what I can tell. The Bayesian aspects in particular are not best suited to a large userbase (as opposed to a single user or small group) and the limited data being used for training, not to mention apparent general reliability issues, lead to a far from technically sound system. Nice idea and all that but ... Whether the 'problem' is down to flawed fundamental principles, bad implementation, bad management or whatever is anyone's guess but either way it does seem far more trouble than it's actually worth for all concerned.
I could just about guarantee you (and be prepared to put money on it !) that if you could set up and train your own filter it would be many times more reliable and more effective than anything PN could offer along similar lines. I know anything I set up would be that's for sure and it most certainly wouldn't take that long to build up an IP black/white list irrespective of whether Bayesian techniques were employed or not. Of course, it's by no means so easy if you rely to a certain extent on webmail or have several users on your A/C as I think you have said before that you do plus it's hassle that we could all live without I'm sure.
I think your 'complaint' is perhaps best not directed at criticalpath at all but rather at PN for not doing what they originally said they were going to do for reasons various. Based on the limited data available, simply redirecting anything tagged by CP straight to that great blackhole in the ether would have produced similar if not better results than Postini if all servers were being fed via CP but with one major benefit: The ability to confirm to your own satisfaction that the system is doing exactly what you want and with little or no side-effects before letting it loose on your data. Yes it is true that there was a very small %ge of false positives initially but it would appear that this issue has been largely sorted out by one means or another ... famous last words !
Regarding image stylee spam, my records show that over recent weeks there has been only a limited and reducing quantity of this type of spam. In previous weeks it often accounted for by far the majority of spam received but last week for instance, a quicky looky see reveals that only around 15 spam messages (<0.3%) contained any image/attachment content of any kind and of those, a significant number were benign images/attachments rather than true advertisements. CP also tagged all but one of those it saw. So far this week (just the first 24 hours obviously) there has only been one message containing images. Postini is not necessarily responsible for your reduction in image spam it may actually be that there simply aren't that many floating around !
As always though, what should perhaps become my standard disclaimer: spam is a very strange, variable, unpredictable and above all distinctly personal thing so YMMV and all that
Moved on to the Postini trial a day or two back. Just logged into Squirrell mail; I now have 1500+ spam emails. Looks like one of my doamin names has been high-jacked - possibly back when PN had the security breach - and the new anti-spam service is not doing as good a job as the old tool of filtering out the trash.
Any comments / advice please?
Any comments / advice please?
The Postini system cannot be worse than the old system because the Postini system filters out obvious spam and then forwards the rest to the old system where the tagging occurs.
Are you saying that the spam folder contains 1500+ emails all received since you switched over to Postini?
Are you saying that the spam folder contains 1500+ emails all received since you switched over to Postini?
What a good idea. I have a very old account john.elvin@lineone.net (Tiscali - shows how old it is) which is still open but I never use. I've just checked and it had 2985 emails since 28/08 sitting in there - the vast majority of which are spam. I've just set it up to forward to a Plusnet mailbox I've just created - lets see how many spams get through to that.
Any idea on how I could get john.elvin@lineone.net on to as many spammers list as possible?
Any idea on how I could get john.elvin@lineone.net on to as many spammers list as possible?
Thanks Phil - I've added that link to the specific topic I started about interpreting the headers.
A query:
I've just received an email from the PUG forums which came through Postini to criticalpath01, then pih-sunmxcore18.plus.net. It has the X-PN-VirusFiltered: by PlusNet MXCore (v2.00) header but no DSPAM headers - why?
A query:
I've just received an email from the PUG forums which came through Postini to criticalpath01, then pih-sunmxcore18.plus.net. It has the X-PN-VirusFiltered: by PlusNet MXCore (v2.00) header but no DSPAM headers - why?