Robots.txt Disallow Question

Discussion in 'robots.txt' started by jda, Nov 28, 2005.

0
  1. #1
    Are there any negatives to blocking all search engines for one day? Could my URL be dropped from a crawl list? I'm expecting heavy traffic tomorrow and don't want an overload because of bots. Thanks.
     
    jda, Nov 28, 2005 IP
  2. Bernard

    Bernard Well-Known Member

    Messages:
    1,608
    Likes Received:
    107
    Best Answers:
    0
    Trophy Points:
    185
    #2
    If the bots return the following day and see the updated robots.txt, it probably won't hurt. It's not something that I would personally do with any site that is ranking and converting well though. A better idea might be to check with your web host and upgrade your hosting account...
     
    Bernard, Nov 28, 2005 IP
  3. ServiceUptime

    ServiceUptime Peon

    Messages:
    42
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #3
    I would never play with that and risk the ranking and everything. Bots might think they are not welcome and leave for good :)
     
    ServiceUptime, Nov 29, 2005 IP
  4. minstrel

    minstrel Illustrious Member

    Messages:
    15,082
    Likes Received:
    1,243
    Best Answers:
    0
    Trophy Points:
    480
    #4
    I don't think it's worth the risk either.

    Ask Brett Tabke how he feels about this issue now :eek:
     
    minstrel, Dec 1, 2005 IP
  5. jda

    jda Peon

    Messages:
    85
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Ended up not doing it, everything was fine. I thought I'd be fine, but I was cool site of the day and know that has caused some to crash.
     
    jda, Dec 2, 2005 IP