cancel
Showing results for 
Search instead for 
Did you mean: 

RE your web page as been archived

Community Veteran
Posts: 26,786
Thanks: 986
Fixes: 10
Registered: ‎10-04-2007

Re: RE your web page as been archived

Quote from: puddy
Their are no hotlinked files on my webpage

How do you know that? I thought the reply by Chris suggested that that was what was happening.
I'm not sure if you understand what hotlinked files are. This is where another website has put a link to a file (image) situated somewhere on your webspace  (note that you may have removed the link to the file from your own web pages so it is only accessible by entering the full URL of the file).
There are two ways to check if any of your files are hotlinked:
  • Look in the log files or webstats
  • Visit every page of every website and every forum in the whole world to see if there is a link to one of your files/images
    Obviously the first is the easier option!  Tongue
  • jelv (a.k.a Spoon Whittler)
       Why I have left Plusnet (warning: long post!)   
    Broadband: Andrews & Arnold Home::1 (FTTC 80/20)
    Line rental: Pulse 8 Home Line Rental (£14.40/month)
    Mobile: iD mobile (£4/month)
    puddy
    Grafter
    Posts: 1,571
    Registered: ‎10-06-2007

    Re: RE your web page as been archived

    if so how come it shows 38000 hits are coming from myself?

    puddy
    Community Veteran
    Posts: 26,786
    Thanks: 986
    Fixes: 10
    Registered: ‎10-04-2007

    Re: RE your web page as been archived

    Can you confirm that you got that from the "Top 30 of nnn Total Referrers" table at the end of the webstats please.
    Are there any other referrers with a high number of hits?
    Over what period was the 38000? You need to look at the stats for just one day when you went over your allowance.
    jelv (a.k.a Spoon Whittler)
       Why I have left Plusnet (warning: long post!)   
    Broadband: Andrews & Arnold Home::1 (FTTC 80/20)
    Line rental: Pulse 8 Home Line Rental (£14.40/month)
    Mobile: iD mobile (£4/month)
    puddy
    Grafter
    Posts: 1,571
    Registered: ‎10-06-2007

    Re: RE your web page as been archived

    3800 hits from me in 3 days and all the hits were for pages that are not hosted. i.e page2/  and not www.puddy.co.uk/page 2
    Plusnet have still not contacted me on this issue yet?

    Puddy
    James
    Grafter
    Posts: 21,036
    Thanks: 2
    Registered: ‎04-04-2007

    Re: RE your web page as been archived

    As far as I can tell, you don't have any open tickets regarding this?
    It's not one of my strengths though, so there's not a lot I can really add to be even remotely helpful, but I'd suggest that raising a ticket regarding this would be a sensible starting point.
    puddy
    Grafter
    Posts: 1,571
    Registered: ‎10-06-2007

    Re: RE your web page as been archived

    Chris Parr said he was going to look in to this problem for me so I did not raise a ticket.  I sent a pm to a staff member and have had no reply
    Still waiting for help on the webstats reporting issue though
    puddy
    gswindale
    Grafter
    Posts: 942
    Registered: ‎05-04-2007

    Re: RE your web page as been archived

    Still worth raising a ticket as that way you're getting official support - sending a PM to a member of staff who may be on holiday or ill means that nobody else at PN can assist as they're not aware of the problem.
    puddy
    Grafter
    Posts: 1,571
    Registered: ‎10-06-2007

    Re: RE your web page as been archived

    Looked at my web stats again 31% are unresolved 
    My partner F9 address has 166 hits  the web site starting www.puddy.co.uk only have a few hits which is correct but again hits on /page2 show 1000's of hits and on the 28th march shows 2381 the site was archived and while the site was archived for a week shows 20451 hits
    What going on?
    I am not that popular!!!  it’s a home web page not a commercial web page
    puddy
    account=mjr99
    mikeb
    Grafter
    Posts: 367
    Registered: ‎10-06-2007

    Re: RE your web page as been archived

    You really want to be ignoring whatever it says on the portal simply because the portal stats, whilst mostly correct, don't tell the whole story. The best thing to do is to process the raw logfiles yourself so that you get a report (just like the one on the portal) but it will also include all direct accesses as well such as to hotlinked files etc. It is then dead easy to spot any problems or unusual activity because you will immediately see large numbers appearing near the top of the various access lists in the report together with exactly who or what is causing them.
    However, you can also check for this by looking at the raw logfiles, it's simply a zipped text file.  There will be a (long) one line entry for each and every access no matter what it was and where it came from. By spotting the 'unusual' or most frequently occurring files or pages accessed and then looking at who is persistently accessing them or where the link came from, you see what the problem is and come up with a solution to it.  It's considerably easier to do than explain how to do mind you !! But each line has the following data on it:
    Quote
    [IP or domain name accessing the file]
    [date and time of access]
    "GET [full path of file] HTTP/1.1" [a 3 digit result code]
    [url of where the link came from]
    [browser type and other info]
    [your_website.co.uk]

    All of this info will be on one single long line looking something like this:
    Quote
    aaa.bbb.ccc.ddd - - [28/Mar/2008:00:16:37 +0000]
    "GET /something/something/image.jpg HTTP/1.1" 200 "http://www.some_website.com/forums/index.php?showtopic=77900"
    "Mozilla/5.0 (Windows; U; Win98; en-GB; rv:1.8.1.12) Firefox/2.0.0.12"
    www.your_website.co.uk

    If you still can't spot the problem then, if you want to, you can PM or mail me the biggest daily logfile you have and I'll have a quick look and (hopefully) tell you what the problem is.  I get no end of problems with monster spikes in usage due to hotlinking (*) and/or stupid people so I'm well used to spotting and dealing with them before they become a problem and PN pull the old plug.
    My guess, however, after a quick look at your site now that it's working again, is that some brain-dead idiot has hotlinked one of your airbus pics from page 2 on a forum, blog, facebook/bebo/myspace page or some stupid AOLer has used it somewhere and the AOL cache/proxy keeps requesting a new copy of it.
    Also, and not wishing to insult anyone's intelligence, but do make sure that you're not using one of the pics as your wallpaper or have the site set as your homepage in your browser and suchlike. I've known more than a few peeps who have accidentally used the file stored on the webserver rather than a local copy and then wonder why they're using so much bandwidth !! Oooops and all that  Roll_eyes

    Off Topic
    (*) Talking of hotlinking as we were, my usual policy is to change the file being hotlinked to a warning requesting that the link is removed from that particular site.  This is because myself and a few selected other peeps regularly hotlink to my files in loads of places quite legitimately so disabling hotlinking in general it is not a sensible option for me.  Sites like myspace, bebo, facebook et al are completely banned regardless because of persistent abuse.
    However, it doesn't end there. If the rogue link is not removed as requested after a reasonable period of time and the file(s) continue to gobble my bandwidth then the offending moron will soon find that the file is no longer quite what they intended it to be and has become something very different. Let's just say it's usually very embarrassing and/or highly inappropriate  Shocked This generally does the trick and the link is removed pretty swiftly thereafter Wink
    So, why have I posted this information I hear you ask ? Simple. Certain PN users (mentioning no names of course) should perhaps be aware of my usual procedure when hotlinking to images and smilies etc. that I have used in my posts on here all over the place. Once or twice is just fine but when the number of hotlinked hits appears on my top 30 list then my fingers begin to get a bit itchy and that can only mean one thing - .hta is about to be edited so someone is about to get a bit of a surprise and feel very silly next time they look at one of their previous posts that contained one or more of my images ! You have been warned  Lips_are_sealed
    puddy
    Grafter
    Posts: 1,571
    Registered: ‎10-06-2007

    Re: RE your web page as been archived

    Thank you mikeB for your input pn are always saying to me if its in webstats it must be true.
    I would be greatful if you could point me in the right direction on how to setup a raw log file,  being a payg account (piggy back the mrs F9 account) pn don't give me any extra platinum type extra's so I am not sure how to keep a log if pn allow it at all?

    Thanks again
    Kind regards
    Michael
    Community Veteran
    Posts: 26,786
    Thanks: 986
    Fixes: 10
    Registered: ‎10-04-2007

    Re: RE your web page as been archived

    Puddy - the first thing you need to do (as advised several times already) is to stop all hotlinking.
    See: http://www.elvin.me.uk/puddys_image.htm
    You should find that in the logs tomorrow without any bother.
    Edit: I'm not surprised people are linking your images - you have some very good ones there.
    jelv (a.k.a Spoon Whittler)
       Why I have left Plusnet (warning: long post!)   
    Broadband: Andrews & Arnold Home::1 (FTTC 80/20)
    Line rental: Pulse 8 Home Line Rental (£14.40/month)
    Mobile: iD mobile (£4/month)
    mikeb
    Grafter
    Posts: 367
    Registered: ‎10-06-2007

    Re: RE your web page as been archived

    PN have apparently enabled your logfiles at long last so you should be able to download them right now via ftp. 
    If you ftp to your webspace you should see 2 directories there now. The first is "htdocs" which contains your site contents and the other is "logs" which contains your raw logfiles. You might also see a 3rd one something a bit like "htdocs.20080119095401" but you can safely just ignore that, it shouldn't be there and PN should have removed it absolutely ages ago.
    If you go into the "logs" directory you will see several files looking something like "your_website.co.uk.[number].gz" these are zipped raw logfiles. There should be one for each day that logfiles have been enabled but limited to a maximum of the last 8 days.  The most recent logfile has no number, yesterdays is "0", the day before "1" and so on. Download them all and unzip them. Pick the largest size unzipped file to have a look at using just about any text editor (wordpad or whatever is just fine) and see if you can spot the problem.
    Of course, it could well be that the logfiles you have right now do not show any problems at all because whatever has been accessing your site hasn't been doing so since logfiles were first enabled.  However, where all this is going to help in the future is that if/when your site is archived again then you WILL have all the necessary data to be able to sort it out because the most recent logfile at that time will (or at least should !) tell you exactly why.
    I'm not going to be round again until very much later tonight but do feel free to mail me the biggest logfile you have and I'll have a look at it when I get back Smiley
    Community Veteran
    Posts: 26,786
    Thanks: 986
    Fixes: 10
    Registered: ‎10-04-2007

    Re: RE your web page as been archived

    In Google do an image search for airbus A380
    You are the 7th highest hit! I'd guarantee your page is going to be archived again tomorrow!
    jelv (a.k.a Spoon Whittler)
       Why I have left Plusnet (warning: long post!)   
    Broadband: Andrews & Arnold Home::1 (FTTC 80/20)
    Line rental: Pulse 8 Home Line Rental (£14.40/month)
    Mobile: iD mobile (£4/month)
    mikeb
    Grafter
    Posts: 367
    Registered: ‎10-06-2007

    Re: RE your web page as been archived

    Well, it looks like Mr.Jelv has
    as per usual !
    Hotlinking is your problem and someone somewhere is stealing your bandwidth. You need some code in a special file called ".htaccess" in your htdocs directory to prevent other sites from directly accessing and using your files, image files in particular.  Something like this springs to mind:

    RewriteEngine on
    RewriteBase /
    RewriteCond %{HTTP_REFERER} !^http://(.+\.)?your_domain\.co\.uk [NC]
    RewriteCond %{HTTP_REFERER} !^$
    RewriteRule .*\.(jpe?g|gif|bmp|png)$ -

    What this should (hopefully !) do is serve the image correctly IFF the request for the image file came from your_domain.co.uk or came from www.your_domain.co.uk or there was no referrer ID because the direct url for the image was typed into the browser etc. In all other cases such as a request coming from www.myspace.com or whatever, the file/image will NOT be served and at best a "you are forbidden to access this page" error message will be seen but generally speaking nothing will be displayed at all if/when an image is hotlinked. That should just about put a stop to it unless someone is being *very* sneaky.
    Before you do anything tho, the above code needs a second opinion because I'm very good at getting .htaccess stuff quite wrong unfortunately and getting it even just a bit wrong can stop your site from working sensibly if not stop it working completely ! If someone can confirm that the above code looks correct (or post a correction) then all you need to do is to change the word "your_domain" to what it actually is, save the code in a file called ".htaccess" and then upload it to your htdocs directory.  There is/was a PN help page somewhere on how to do all this but I can't find it now and in any case, I think it actually contained duff info the one and only time I found it and tried to use it ... although that was a good few years ago I must add.
    If you click Mr.Jelv's previous link before you do this you will see that he has a webpage that hotlinks to one of your images.  If you do the same thing again after uploading .htaccess (and clearing your browser cache) then you should not see your image on his webpage at all. If you then check your own webpages, they should not be affected in any way and all images should display as normal.
    FWIW, I've just had a quicky poke around and your image called "media_object_image_lowres_A380_Cold_snow_mr.jpg" apparently appears on this forum page as well as on a few other pages on the same site it would seem.  You have to be registered with the forum to actually see the pic and I'm not but all the indications are that some users on that particular forum are hotlinking to your image(s). It has also been hotlinked by users on this Polish forum as well. This almost without doubt explains the extra traffic you've been getting.
    Most excellent pics btw, I might just have to be 'borrowing' a copy of some of them Wink
    Superuser
    Superuser
    Posts: 3,810
    Thanks: 2,264
    Fixes: 14
    Registered: ‎10-04-2007

    Re: RE your web page as been archived

    Was this the link Mike?