crawl rate in robots.txt

Discussion in 'Search Engine Optimization' started by medrastic, Apr 14, 2008.

0
  1. #1
    does crawl rate really helps the bot to visit the site frequently. Is it good to put the crawl rate permanently fastest?
     
    medrastic, Apr 14, 2008 IP
  2. johncena000

    johncena000 Banned

    Messages:
    460
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    0
    #2
    I think.. it will put more load on the server if it's selected fast..
     
    johncena000, Apr 14, 2008 IP
  3. Loy Maben

    Loy Maben Peon

    Messages:
    298
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Loy Maben, Apr 15, 2008 IP
  4. enous

    enous Well-Known Member

    Messages:
    1,500
    Likes Received:
    16
    Best Answers:
    0
    Trophy Points:
    158
    #4
    Loy Maben,good post, thanks for sharing.
     
    enous, Apr 15, 2008 IP