Questions about robots.txt...

Discussion in 'robots.txt' started by White Monkey, Aug 22, 2006.

0
  1. #1
    Ok I have a few questions about robots.txt

    1.Is it good to have robots.txt on your website with adsense?
    2.Have I got robots.txt on my website? (how do I check?)
    3.How do I take it oof if it is bad?

    Thanks
     
    White Monkey, Aug 22, 2006 IP
  2. Chuckun

    Chuckun Well-Known Member

    Messages:
    1,161
    Likes Received:
    60
    Best Answers:
    2
    Trophy Points:
    150
    #2
    1. Not so much for AdSense, but for site crawlers, robots.txt is great.
    2. http://yoursite.com/robots.txt or look in your root directory for the file robots.txt
    2. delete the file: robots.txt
     
    Chuckun, Aug 22, 2006 IP
  3. White Monkey

    White Monkey Peon

    Messages:
    260
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #3
    My site is a blog hosted with blogger. Do they have robots.txt by default or what?
     
    White Monkey, Aug 22, 2006 IP
  4. Gatorade

    Gatorade Peon

    Messages:
    2,130
    Likes Received:
    222
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Check out www dot searchtools dot com/robots/robots-txt.html (Replace the "dot" with actual .). It gives you a better understanding of robots.txt.
     
    Gatorade, Aug 22, 2006 IP
  5. White Monkey

    White Monkey Peon

    Messages:
    260
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #5
    All I want to know is. Does blooger blogs come with robots.txt as default?
     
    White Monkey, Aug 22, 2006 IP
  6. makeit easy

    makeit easy Active Member

    Messages:
    2,067
    Likes Received:
    62
    Best Answers:
    0
    Trophy Points:
    90
    #6
    You should have a yourblog.blogspot.com/robots.txt sub-directory when you check. Then you should upload robots.txt file. That's my problem, too. I can't upload this file since I can't access the host.
    I think there is only one solution to upload robots.txt file: There must be a host which a blog owner can control it.
     
    makeit easy, Aug 29, 2006 IP