How to dodge web spiders/bots

Discussion in 'Traffic Analysis' started by adzeds, Feb 10, 2010.

  1. #1
    I am writing a PHP script that collects web stats for one of my website. I have created a function that checks the user agent to locate whether it is a bot or real person on my site.

    What I need is a list of all the bots and spiders so that I can block them. Anyone know where I can find one, keep searching the net but keep getting info about the biggest spiders in Australia which is not helpful..

    Hope someone can help!
     
    adzeds, Feb 10, 2010 IP
  2. adzeds

    adzeds Well-Known Member

    Messages:
    1,209
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    100
    #2
    I am looking for something like a csv file with names of search engine spiders and web bots
     
    adzeds, Feb 11, 2010 IP