way back machine

Discussion in 'All Other Search Engines' started by USPB, Oct 31, 2007.

0
  1. #1
    do you block this in your robots.txt?
     
    USPB, Oct 31, 2007 IP
  2. Divisive Cottonwood

    Divisive Cottonwood Peon

    Messages:
    1,674
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    0
    #2
    No, why!

    er... I can't think of a reason why you should.

    I've never noticed much bot traffic from this resource.

    wayback is :cool:

    are you on a thread starting mission at DP USPB? ;):p
     
    Divisive Cottonwood, Oct 31, 2007 IP
  3. Pixelrage

    Pixelrage Peon

    Messages:
    5,083
    Likes Received:
    128
    Best Answers:
    0
    Trophy Points:
    0
    #3
    no way! That site is a great legal reference if you want proof that your site has been on the internet since you first put it up.
     
    Pixelrage, Oct 31, 2007 IP
  4. USPB

    USPB Peon

    Messages:
    887
    Likes Received:
    13
    Best Answers:
    0
    Trophy Points:
    0
    #4
    thanks for the advice
     
    USPB, Oct 31, 2007 IP
  5. thenetninja

    thenetninja Peon

    Messages:
    314
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    #5
    you can effectively block the web archive bot through htaccess or even php, can't be sure of the user agent it shows put it shows up in your stats
     
    thenetninja, Oct 31, 2007 IP