Is anyone else having trouble with Trend Micro's bots? I don't really want to have to block them, because they're probably well intentioned, but I think I'm going to have to because they're just so stupid.
These are the bots that follow you round if you're running a Trend Micro safe-browsing product. Any time you visit a site, a couple of minutes later the Trend Micro bots come trotting along in your exact footsteps, like a pair of little dogs, presumably baiting sites to push them malware. Cute.
One problem I have with them is that they have your query string, so one of my search interfaces is getting hit two or three times for each real search a Trend Micro user conducts. (Incidentally, I can't use POST, because users need to be able to link to their search results.) Search stats are used in development and the Trend Micro bots are skewing them.
Another problem I have with them is that they hit parts of my sites to which they have no access, so they're just getting a login form returned, and that just wastes bandwidth. They don't respect robots.txt.
Another problem is one of informed consent. A Trend Micro user with whom I discussed this said she had no idea she was being followed round like this and wasn't entirely comfortable with the idea. When I asked Trend Micro if one could switch this feature off, they replied
|You can disable Smart Surfing|
followed, a few minutes later, by
|Sorry for my mistake. There's an option during the installation. You can tick off the box "share information with Trend Micro".|
So you have to opt out at installation, without really knowing what information you would be sharing with Trend Micro. The user mentioned above had had the software pre-installed on her new PC as a free trial. Looking at the blurb, all she could find was
|Smart Feedback is a type of mechanism that detects the latest threats and reports them to the Trend Micro Smart Protection Network.|
which she didn't think conveyed the fact that her browsing history was routinely being shared with robots in Japan and the US.
At this point I should probably plug ZoneAlarm and Little Snitch (outbound firewalls).
I also asked Trend Micro why their bots didn't respect robots.txt and was told
|This is not a robot. This is the way the software works.|
Sure look like robots to me. They present as "MSIE 6.0; Windows NT 5.1", i.e. stupidly vulnerable - presumably to solicit attacks. But malware sites would choose old-fashioned methods to attack that sort of profile (why waste effort?), so all this trawling round is presumably missing more recent techniques. It's probably well-intentioned, but I wouldn't call it smart.
Consign to bad-bot category, I guess.