wat if i set the Crawl rate control to Faster??

Discussion in 'Google Sitemaps' started by bullzeye, Jun 28, 2007.

  1. #1
    wat wil happen if i switch the Crawl rate control in sitemaps from normal to faster.. wil it make any harm??
     
    bullzeye, Jun 28, 2007 IP
  2. zehlendorf

    zehlendorf Member

    Messages:
    93
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    48
    #2
    This will put more load on your server. However unless your server is currently on the brink of losing it then it shouldn't matter.
     
    zehlendorf, Jun 28, 2007 IP
  3. madk

    madk Peon

    Messages:
    141
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Not to hijack...but none of my sites have the option of selecting a high craw rate speed. Anyone knows how Google decides this?l
     
    madk, Jun 28, 2007 IP
  4. celebs101

    celebs101 Peon

    Messages:
    64
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #4
    because the number of pages you have in your site are already probably indexed by Google so it does'nt need to crawl more pages on your site that's the reason you do not have faster-crawl option available.

    like i've got over 100,000 pages and google has only indexed few thousand pages, by enable the faster crawl option it will just index he 100,000 pages faster. and I did measure the Google Bot comparing before setting it to faster and to be frank I did'nt notice huge bandwidth jump.. hardly 15% from google bot increased.
     
    celebs101, Jun 30, 2007 IP
  5. Aragorn

    Aragorn Peon

    Messages:
    1,491
    Likes Received:
    72
    Best Answers:
    1
    Trophy Points:
    0
    #5
    That option will only be enabled if google is your site is very slow and google is not able to crawl it within it a few days. If you don't see that option, then be happy that Google is not finding any problem with your site. If you set the crawl rate to faster, then google will start requesting for more number of pages at the same time. This can have adverse effect on your server load and bandwidth.
     
    Aragorn, Jul 2, 2007 IP
  6. trichnosis

    trichnosis Prominent Member

    Messages:
    13,785
    Likes Received:
    333
    Best Answers:
    0
    Trophy Points:
    300
    #6
    it will not harm your web site. google is also suggesting you crawl speed to avoid from hurting your site
     
    trichnosis, Jul 2, 2007 IP
  7. bullzeye

    bullzeye Guest

    Messages:
    154
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #7
    i asked this question cuz my new updated home page was not cached for a long time.. i set the crawl rate to faster then after 2 days the page was indexed.. i dunno whether it was a normal cache or it was becoz i changed the crawl rate.. After it got cached i turned the crawl rate to normal again.
     
    bullzeye, Jul 10, 2007 IP
  8. permaximum

    permaximum Active Member

    Messages:
    43
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    91
    #8
    Google crawls your pages a little bit faster.....
     
    permaximum, Jul 13, 2007 IP
  9. infonote

    infonote Well-Known Member

    Messages:
    4,032
    Likes Received:
    68
    Best Answers:
    0
    Trophy Points:
    160
    #9
    It will hit your server especially if you are in shared hosting. My site was disabled for 24 hours because "I was hit by Yahoo and Google bots at the same time".
     
    infonote, Jul 13, 2007 IP
  10. Madmurphy

    Madmurphy Active Member

    Messages:
    341
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    53
    #10
    Where do they have this option at?
     
    Madmurphy, Jul 15, 2007 IP
  11. humble.techy

    humble.techy Peon

    Messages:
    16
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #11
    Hope you signed up for Google Webasters, otherwise go to http://google.com/webmasters and signup. Then verify your site by adding a meta tag or uploading html page. Then if you go to Diagonistic and click on 'Crawl rate' under Tools on the left side.
     
    humble.techy, Jul 20, 2007 IP
  12. InfiniteTech

    InfiniteTech Active Member

    Messages:
    380
    Likes Received:
    7
    Best Answers:
    0
    Trophy Points:
    70
    #12
    To be ranked high on Google, and other SEs you need to have very actively updated pages. If you increase the reindex rate, you are just doing more harm than good, SE ranking for that page will drop down as it is has a *stale* content.

    That said, do not extend the reindex to a year or something, then you will have un-targeted traffic.
     
    InfiniteTech, Jul 20, 2007 IP