Where should I place robots.txt and XML sitemap on the site?

Discussion in 'Search Engine Optimization' started by zan15, Sep 22, 2009.

0
  1. #1
    I have a site, all its pages are secure. I have created robots.txt and XML sitemap. Where should I place them? In the public directory or secure?

    Please advise.
     
    zan15, Sep 22, 2009 IP
  2. chodom

    chodom Peon

    Messages:
    3
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    I think you have to put it in the public_directory, it's easy for bot crawl your website.
     
    chodom, Sep 22, 2009 IP
  3. monfis

    monfis Well-Known Member

    Messages:
    1,476
    Likes Received:
    31
    Best Answers:
    0
    Trophy Points:
    160
    #3
    Root level - (where your index page is placed).
     
    monfis, Sep 22, 2009 IP
  4. zan15

    zan15 Peon

    Messages:
    49
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    My index page is itself a secure page. In that case, do you suggest to put robots and sitemap in the secure directory?

    Like chodom, even I was thinking that if I place both both robots and sitemap in public directory it will be easy for the crawler to crawl. But I am not sure.
     
    zan15, Sep 22, 2009 IP
  5. Canonical

    Canonical Well-Known Member

    Messages:
    2,223
    Likes Received:
    141
    Best Answers:
    0
    Trophy Points:
    110
    #5
    The robots.txt file HAS to go in the root folder of your web (i.e. http://www.example.com/robots.txt). The sitemap.xml can go anywhere on your site since you can specify the complete path inside your robots.txt or when you manually submit it to the engines.
     
    Canonical, Sep 22, 2009 IP
  6. doganmanis

    doganmanis Peon

    Messages:
    57
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #6
    place in your main index page,
     
    doganmanis, Apr 6, 2010 IP