CometQ Bot

Discussion in 'All Other Search Engines' started by TVCokeCan, Oct 25, 2006.

  1. #1
    Due to the fact CQ bot has just got a whole new Cluster server we are going all out to get to the 2 million mark, and thus fear not if you get User agent "CometQ Bot". The bot is not gathering anything except website discription title and keywords. You can block the CQ Bot by creating a file called robots.txt and insert this into it:

    User-agent: CometQ Bot
    Disallow: /
    Code (markup):
    However that also means you wont get indexed and might lose out on traffic. If anyone has any questions about the bot just ask. Also if you don't want it to crawl but want it to be able to find your pages, you can use Google Sitemaps, just add this into your robots.txt file

    User-agent: CometQ Bot
    Disallow: /
    Sitemap: sitemap.xml
    Code (markup):

     
    TVCokeCan, Oct 25, 2006 IP