1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

How do i tell when google crawls my site?

Discussion in 'Search Engine Optimization' started by helix@ssc, Oct 15, 2009.

  1. #1
    Is there a way to tell when the last time google crawled my site? It seems like i have to wait a week or so for the latest changes that I make to my site start making a difference on google search.

    www.silverstarcabinets.com
    www.card-racks.com

    Thanks for your help.
     
    helix@ssc, Oct 15, 2009 IP
  2. brokencode

    brokencode Active Member

    Messages:
    138
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    53
    #2
    submit your sitemap to google webmaster tool.
     
    brokencode, Oct 15, 2009 IP
  3. theapparatus

    theapparatus Peon

    Messages:
    2,925
    Likes Received:
    119
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Your webserver logs should tell you when the last time you had a visit from the Googlebot. You can scan for the browser agent of Googlebot although it may be fake. This post should help you tell if it's real or not:

    http://googlewebmastercentral.blogspot.com/2006/09/how-to-verify-googlebot.html

    The Googlebot will also check the robots.txt file on every visit as well.

    To be honest though, Googlebot scans most of my sites each and every day. The webmaster tools has a page within it that'll show you how many pages a site gets indexed each day.
     
    theapparatus, Oct 15, 2009 IP
  4. trosquin

    trosquin Active Member

    Messages:
    681
    Likes Received:
    9
    Best Answers:
    0
    Trophy Points:
    60
    #4
    yup Google webmaster Tools is the best....but you can also type this into a google search

    cache:www.silverstarcabinets.com

    this will give you a cached version of your site with a date....this will be the last time your site was crawled.
     
    trosquin, Oct 15, 2009 IP
  5. speakeasy602

    speakeasy602 Active Member

    Messages:
    150
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    51
    #5
    Actually, Google says that if you set your crawl rate, they cannot guarantee the crawl frequency, plus this way will actually hinder your site's visibility more than help it. I use webmaster tools to see how often my site is crawled, it gets crawled every day, so I don't monitor it as much anymore.
     
    speakeasy602, Oct 15, 2009 IP
  6. me_king

    me_king Peon

    Messages:
    83
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #6
    Add an XML sitemap

    It can take quite some time for Google’s crawlers to index all the pages in a new website just by following links.

    The larger the website, the more time it can take. Pages at a high click depth from your homepage can take a lot longer to get indexed because the crawlers don’t discover them until after several rounds of indexing and link following have occurred.

    I find that adding an XML sitemap really solves this problem because it tells Google about all your pages ahead of time.

    If you have a large website with many high click depth pages then an XML sitemap will help indexing enormously.
     
    me_king, Oct 15, 2009 IP
  7. MayaLocke

    MayaLocke Peon

    Messages:
    1,016
    Likes Received:
    14
    Best Answers:
    0
    Trophy Points:
    0
    #7
    Create an Google Webmaster Tools account and you can look at the "chart" to see when your site is spidered.
     
    MayaLocke, Oct 20, 2009 IP
  8. CaboChris

    CaboChris Peon

    Messages:
    79
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #8
    He's using google analytics......in your webmaster tools!
     
    CaboChris, Dec 10, 2010 IP
  9. haasan

    haasan Active Member

    Messages:
    226
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    51
    #9
    Google's crawling will be traced by google webmaster tool. But not necessarily all crawled links will be indexed by Google. Hope you understand the difference between crawling and indexing.

    SJ
     
    haasan, Dec 10, 2010 IP
  10. instantit

    instantit Peon

    Messages:
    124
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #10
    Make use of Google webmaster tools and eventually you will find it useful on your everyday tracking and updates.
     
    instantit, Dec 10, 2010 IP
  11. eoadonia

    eoadonia Peon

    Messages:
    43
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #11
    Submit XML site map to the Google webmaster tool is the best way to tell the Google about the pages to crawl.
     
    eoadonia, Dec 10, 2010 IP