bots and my site

Discussion in 'robots.txt' started by Scarface., Jul 26, 2006.

0
  1. #1
    hello,
    my site is up for about three days now and im just wondering if this is good or bad...

    [​IMG]
     
    Scarface., Jul 26, 2006 IP
  2. ian_ok

    ian_ok Peon

    Messages:
    551
    Likes Received:
    11
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Always best to check your logs, as not ALL bot request your robots.txt and therefore whon't showon this list.

    I've banned alexa (ia_archiver) in the htaccess, the 2nd one you will have to check your text logs MSN & Google are of course well know to all.....even you?

    Ian
     
    ian_ok, Jul 26, 2006 IP
  3. dagg3r

    dagg3r Peon

    Messages:
    92
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #3
    shouldnt be a problem they are probably indexing or something i had a similar thing on my logs too on one of my newer domains
     
    dagg3r, Jul 31, 2006 IP