Question about htaccess and robots.txt for my site

Discussion in 'Site & Server Administration' started by indyonline, May 3, 2011.

  1. #1
    I have a site that has wordpress in the root folder and I have SMF forum in my /forum folder and a PHPLD link directory in my /directory folder.
    I was wondering if I can have an htaccess file for each and have them in each directory or if I should only have one in the root of my site.
    If I should only have one, what should I include for the fourm and directory?

    I have read I should only have one robots.txt file. assuming this is true, can anybody help me to create the robots file I would need? or tell me what to include.

    any input would be appreciated and I thank you in advance.

    Thanks, Rob
    ----------------------------

    Edit:
    One last question, Is it ok to have multiple XML sitemaps submitted in Google webmaster tools? I have not found a sitemap generator that will create a good site map for my situation so I have a sitemap for each one individually and have submitted them all to webmaster tools.
    thanks again.
     
    indyonline, May 3, 2011 IP
  2. dvduval

    dvduval Notable Member

    Messages:
    3,372
    Likes Received:
    356
    Best Answers:
    1
    Trophy Points:
    260
    #2
    Yes, google webmaster tools makes it easy to have multiple sitemaps, and that is not a big deal, but the most important thing is your URLs should be easily spiderable (without the sitemap). Both wordpress and phpLD do well with this already without any action needed on your part. As for robots.txt, unless you are having a problem with spiders behaving badly, you probably will be fine if you don't even worry about robots.txt
     
    dvduval, Jun 19, 2011 IP