Doing Crazy Stuff You Don't Want People to See Later?

Discussion in 'General Business' started by MrX, Jun 1, 2007.

  1. #1
    I was recently asked about this by one of our clients. They were experimenting with some unique things on their site and didn't it want it traceable down the line. Here's the solution:

    Make sure you exclude archive.org (the way back machine) using your robots.txt. Just stick this in your robots.txt file:

    User-agent: ia_archiver
    Disallow: /
    
    Code (markup):
    and archive.org will delete any records it has of your site, and won't cache it in the future (until you remove that code).

    This is common knowledge to most of us, but a lot of webmasters haven't even heard of archive.org so it might be news to some.

    Just a tip. :)
     
    MrX, Jun 1, 2007 IP
  2. turiel

    turiel Peon

    Messages:
    148
    Likes Received:
    11
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Thanks for that info, it never occured to me to block things from the Internet Archive.

    However, I am almost sure that Google stores previous versions of webpages and will be launching a service similar to archive.org.

    If you're doing "crazy stuff" on your site... wouldn't you just password protect it?
     
    turiel, Jun 1, 2007 IP
  3. MrX

    MrX Well-Known Member

    Messages:
    1,563
    Likes Received:
    77
    Best Answers:
    0
    Trophy Points:
    140
    #3
    I'm sure Google would have a similar opt-out feature.

    As for crazy stuff, I just mean things in general that you don't want to be archived forever and forever.
     
    MrX, Jun 1, 2007 IP
  4. Kontent.solutions

    Kontent.solutions Peon

    Messages:
    1,157
    Likes Received:
    59
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Being archived works both ways. Good and may be at times negative too.

    Googles cache feature is pretty much not the same, but then it does give you an idea what it was earlier. So you can expect them to come up with something like that in the near future.

    ~G
     
    Kontent.solutions, Jun 1, 2007 IP
  5. Pixelrage

    Pixelrage Peon

    Messages:
    5,083
    Likes Received:
    128
    Best Answers:
    0
    Trophy Points:
    0
    #5
    archive.org helped to prove many of my non-registered trademark stuff. I don't think I'd ever want to block it!
     
    Pixelrage, Jun 1, 2007 IP
  6. Game Producer

    Game Producer Well-Known Member

    Messages:
    1,996
    Likes Received:
    158
    Best Answers:
    0
    Trophy Points:
    160
    #6
    I have a better idea, don't put in public that crazy stuff you don't want other people to see ;)

    Eh, thanks for the tip.
     
    Game Producer, Jun 2, 2007 IP