robots.txt????

Discussion in 'WordPress' started by BiafranMan, Aug 10, 2008.

0
  1. #1
    i just signed up for the google webmasters tool and its says:

    "URLs restricted by robots.txt: 22"! including my main page!

    y? i never made a robots.txt, does wordpress make one when you install??where is it so i can edit it!?
     
    BiafranMan, Aug 10, 2008 IP
  2. brettk

    brettk Member

    Messages:
    811
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    33
    #2
    that doesn't seem right. I'm not sure if I understand your question but basically robots.txt is somethign that you make so that search engines don't crawl that page so it doesn't get indexed.
     
    brettk, Aug 10, 2008 IP
  3. BiafranMan

    BiafranMan Peon

    Messages:
    119
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    i know! and i never made one but googles webmasters tool says a robots.txt is stoping them from crawling 22 urls! one being my main page!
     
    BiafranMan, Aug 11, 2008 IP
  4. JSellnau_TSCC

    JSellnau_TSCC Peon

    Messages:
    166
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Look on your hosting and if there is a robot.txt there delete it.
     
    JSellnau_TSCC, Aug 11, 2008 IP
  5. BiafranMan

    BiafranMan Peon

    Messages:
    119
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    i did! i cant find one!
     
    BiafranMan, Aug 11, 2008 IP
  6. brettk

    brettk Member

    Messages:
    811
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    33
    #6
    did you check your ftp of the site? It would be in the main folder I believe.
     
    brettk, Aug 11, 2008 IP