1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

How bad is it to NOT have a robots.txt file

Discussion in 'robots.txt' started by PuneetJvw, Jan 16, 2010.

0
Page 2 of 2
  1. travelguides

    travelguides Peon

    Messages:
    107
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #21
    If you don't have one, it could generate a 404 error and redirect a search engine to your default 404 error page and confuse it. No harm in having one and it is in fact advantageous to have one.
     
    travelguides, Jul 25, 2010 IP
  2. sunnykwok25

    sunnykwok25 Peon

    Messages:
    33
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #22
    It can complement your sitemap, there is a code you can put in robots.txt where it helps the search engine locate your sitemap.
     
    sunnykwok25, Jul 28, 2010 IP
  3. smartamit04

    smartamit04 Peon

    Messages:
    97
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #23
    Hi,
    It is just for search engine to prevent any page of directory to crawl. it doesn't have any bad effect.
     
    smartamit04, Aug 4, 2010 IP
Page 2 of 2