htacess rewrite impact on robots.txt

Discussion in 'Apache' started by JohnnyCashpoint, Dec 12, 2008.

  1. #1
    Hi everyone. I'm new here.

    I have it on good authority that this is the place to get accurate advice on all things of this nature so here goes.

    I've just created a site in php.

    I have a rewrite rule in my .htacess file to rewrite all my .php files to .html

    When creating my robots.txt, what extension do I reference when allowing / disallowing robots (the original .php or the rewrite .html, or both)?

    If I had a function (e.g. function.php) would I need to disallow that too - if so, would I reference it as .php or .html (because of my htaccess rewrite rule) ?

    One last question. Is it wise move to put all my includes in an /include/ sub folder and disallow that too ?

    Best regards and thanks in advance.
    Lee.
     
    JohnnyCashpoint, Dec 12, 2008 IP
  2. hexmax

    hexmax Peon

    Messages:
    44
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    you can use disallow for .html files and disallow /include will prevent robots from access inside
     
    hexmax, Dec 12, 2008 IP