cancel
Showing results for 
Search instead for 
Did you mean: 

Website timeouts

mentalist3d
Grafter
Posts: 371
Registered: 20-08-2009

Website timeouts

I have a sub-domain at art.concepts.org.uk which is a free online art portfolio for people to promote their artistic skill, recently I noticed I was getting a lot of traffic sent to me from spammy sites for cigarettes and herbal viagra. When I visited the pages that were sending me the traffic, I couldn't find my link, so I reckon it was something dodgy and banned the websites IP addresses. Since then my site has started to receive higher than normal timeouts (its now about a couple of times per day rather than 1 or 2 per fortnight). So I am just wondering if my site is under some form of attack from these malicious sites, and if so, can anyone point me to any online articles for securing my site against these attacks?
3 REPLIES
Community Gaffer
Community Gaffer
Posts: 12,966
Thanks: 753
Fixes: 70
Registered: 04-04-2007

Re:Website timeouts

How exactly have you 'banned the IP address'? Whilst it's geared towards our Homepages/CGI platforms, this might be of some interest.

Bob Pullen
Plusnet Products Team
If I've been helpful then please give thanks ⤵

mentalist3d
Grafter
Posts: 371
Registered: 20-08-2009

Re: Website timeouts

I should have mentioned that its externally hosted (not on PlusNet servers), apologies.
The IP addresses have been banned via cPanel which is installed on the server. Thanks for the link, looking at the info, I reckon I'll need to create a robots.txt for the site, and see how that goes as there is a high number of unknown robots crawling the site on a daily basis.  Thanks again, hopefully this will help Smiley
mentalist3d
Grafter
Posts: 371
Registered: 20-08-2009

Re: Website timeouts

I've now written a robots.txt for my site, but whilst researching info on this, I did find that bad bots usually ignore the robots.txt, however I have found a workaround:
Quote
Add a line like this to your robots.txt file:
Disallow: /email-addresses/
where 'email-addresses' is not a real directory. Wait a decent amount of time (a week to a month) then go through your access_log file and pick out the User-Agent strings that accessed the /email-addresses/ directory. These are the worst of the worst - those that blatantly disregard our attempts to keep them out

The full article can be seen at: http://www.evolt.org/article/Using_Apache_to_stop_bad_robots/18/15126/