Just wondering what you think - the GoogleBot has made thousands of pageviews on my site over the last week or two, since the site has been live, and there are about 1500 pages in the index. However, there is next to zero search traffic (after a brief surge the first day or two). What's been your experience - would Google bother visiting all those pages (product stuff) if it is going to dump my site in the SERPS? Cos I don't want the GB sucking up my bandwidth (and API calls) if I'm not going to see any results! Anyone else shed light on this behaviour?
So you have a website with a lot of pages that are inter-linked. It's normal for G-bot to come and cache as many as it can. How many it caches/indexes it's based on your website content (unique?) and on the amount of inbound links to different pages. All that indexes should not drop your website in SERPS. As for bandwidth, if your website is PHP/ASP or some other server side scripting, you can gzip the output (HTML). Most browser and search engines know how to decrypt gzipped content and that should lower your traffic drastically. If you have a lot of text, this should save up to 70-80% of the bandwidth.
In general the more you see the GoogleBot, the better so i think this is a good thing. As for your bandwidth, you can use GoogleWebmaster Tools to specify a slower crawl-rate - but if it was me I would not do anything to discourage the little Gbot..
Sure - actually I am not so worried about server bandwidth as about the hundreds of API calls that get made to the retailers I am featuring. I am not caching yet, there is no point because Google is practically the only visitor, so no-one else to cache for. But I am just scratching my head thinking, where's all the traffic?
The more content, the better. On previous sites I had built, the more posts I had, the quicker the G-bot crawled my site.
how old is your domain ? if its not too old then you have to wait a little bit - bcoz the age factor is important in the eyes of G .But yeah the keywords are the most important compared to age .be very patient .
Now since one month only google started crawl regulerly. Any how its better for regulerly updating site. Also they change some algorithm, may be plans to do some in the web ranking. i hope it will be +ve effect , also little scared for negative result
Yeah, it's pretty new, like only a month or so. I can imagine a site appearing out of the blue with tons of content doesn't necessarily make Google jump for joy! Getting just a tiny trickle of Google traffic the last day or so - hopefully it will turn into more than a trickle, since Google now says 1650 pages indexed.
I never check how many times Google bot is reading my site. But since you people said that it is a good thing that Google bot come many times and I just build a new site, I will check how many times the bot visit my new site(just 2 days old now)
Well, I use a neat little free PHP stats program called TraceWatch - you can find it easily if you want - which is really handy for a new site because it shows you clearly every single visit, including bot visits and everything else. So while the site is still new, you can see what's going on. After a while you can move over to some client-side solution like Google Analytics or whatever, to stop your server getting bogged down. I still have yet to see if 15,000 bot visits in the last two weeks IS a good thing though, i.e. is it going to translate into search traffic...!
Google,yahoo,msn bots are visiting daily to get new content from your website and put to search engines it is normal thing and you should not ignore this bots,all what they want is just fresh information from your website