1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Is it necessary to upload a robots.txt file to the website?

Discussion in 'Search Engine Optimization' started by Anjali Gupta, Aug 23, 2013.

  1. #1
    I have a new website, Can anyone suggest that is there compulsion to upload a robots.txt file to my website, If i don't want to disallow any page?
    Anjali Gupta, Aug 23, 2013 IP
  2. royalstar_23

    royalstar_23 Active Member

    Messages:
    219
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    90
    #2
    Well, Its not a compulsion..But it guides search engine as which directory he should not view, For example in our website we have config folder, Log folder, Temp folder.... I would like to google to crawl that too from my website so i would preferably direct the google/search engines to leave those directories from the website..

    Hope my answers is clear to you.. Let me know if you need any assistance.

    Regards
    Vismit
    royalstar_23, Aug 23, 2013 IP
  3. Kianu42

    Kianu42 Member

    Messages:
    511
    Likes Received:
    15
    Best Answers:
    5
    Trophy Points:
    43
    #3
    Uploading robot.txt file is a must have for a better crawlability of your site and thus improving its onpage optimization.
    There is a plenty of free robot.txt generators and there is one in GWT.
    Kianu42, Aug 23, 2013 IP