cancel
Showing results for 
Search instead for 
Did you mean: 

Spiders

N/A

Spiders

I have quite an old style web site with lots of text on each page. I have managed to get search engines like Google to list it's content but it only seems to list the first half of the text. Does any one know if the spiders like googlebot have a maximum amount of text per page that they will list. Or will they come back and get the rest later?
4 REPLIES
N/A

Spiders

My site is basically text-only and was designed in its topology before Google existed. Most of the files are at the 4th level so Google doesnt pick up very many of them. Of those it does pick up I hadnt heard of only part of the contents being picked up by the spiders, which are trawling me all the time. There is a very good 'Google News' at webmastersworld.com where you can find out what is happening in Google and ask questions.
N/A

Spiders

http://www.webmasterworld.com/forum3/

is where to find out.

I've removed the double post for you which I'm sure was a slip of the mouse (johnessex forum moderator)
N/A

Spiders

Thanks Douglas I'll give that a go.
N/A

Spiders

I have seen the 'Google News' webmasters speculating that there might be a limit of 100K on what a spider can absorb, but I have never seen them mention only part of a file being taken.