Googlebot in AWstats vs. Googlebot in sitemaps?

Discussion in 'Google Sitemaps' started by solid7, Aug 21, 2006.

0
  1. #1
    OK, someone PLEASE explain this to me. When I first put my site up, it was getting spidered like crazy. I got like 350+ hits per month, with no real effort. Probably, this was due to the fact that it was a new site, XHTML compliant, and was pinging with every new page posted. After the initial frenzy, I noticed that the Googlebot visits dropped off. Again, no problem, one would expect this. However, now, I've just added a sitemap, and my Googlebot visits are frequent again - about 20 per day. However, when I look in my sitemaps account, I see this message:

    The date is totally inconsistent with what I'm seeing in my stats.

    Can someone please explain what the difference in these Googlebot visits is?

    Thank you.
     
    solid7, Aug 21, 2006 IP
  2. websitetools

    websitetools Well-Known Member

    Messages:
    1,513
    Likes Received:
    25
    Best Answers:
    4
    Trophy Points:
    170
    #2
    1. Someone may be disguising themselves as GoogleBot in user agent.
    2. There may pass some time before all Google datacenters with all their GoogleBots update their data into Google sitemaps
     
    websitetools, Aug 22, 2006 IP
  3. solid7

    solid7 Well-Known Member

    Messages:
    459
    Likes Received:
    51
    Best Answers:
    0
    Trophy Points:
    148
    #3
    Is 7 days enough?

    My last recorded crawl was on the 15th. The Googlebot has been to my site over 50 times since then, and a whole bunch of pages got indexed. Am I not understanding what is a "Crawl" and what is just a "visit"?

    No, it's not a fake referrer - the Googlebot traffic coincides precisely with the creation of the sitemap.
     
    solid7, Aug 22, 2006 IP
  4. Jean-Luc

    Jean-Luc Peon

    Messages:
    601
    Likes Received:
    30
    Best Answers:
    0
    Trophy Points:
    0
    #4
    In his first video (July 31), Matt Cutts (Google) said that the "pieces of data" within sitemaps are updated at different times and different frequencies. Some parts are updated on a daily basis, for others it can take weeks.

    I trust AWStats.:)

    Jean-Luc
     
    Jean-Luc, Aug 22, 2006 IP
  5. Vanessa Fox

    Vanessa Fox Peon

    Messages:
    14
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #5
    The last crawl date is specifically referring to the home page. It sounds like may need to do a better job of making that clear. (At the moment, we only show you the most recent crawl date of other pages in your site if we had trouble crawling them -- on the crawl errors page).
     
    Vanessa Fox, Aug 23, 2006 IP
  6. Wiley

    Wiley Peon

    Messages:
    127
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #6
    It is only the HOME page that they are refering to. Also right now many features have stopped working like "Prefered Domain" and "Crawl Stats".

    Vanessa what would really help is to have an area on your home page telling us what you are working on so we don't find our selves trying to fix things like "Prefered Domain" when Google is hopefully making it better. Can be very frustrating trying to set the "Prefered Domain" option again and then find out it is not even working and not knowing if this is something we are going to have to set or not in the future.
     
    Wiley, Aug 23, 2006 IP