Need help controlling bots (robots.txt - htaccess - etc)

Discussion in 'robots.txt' started by mdvaldosta, Dec 1, 2005.

0
  1. #1
    Ok, here's the deal. I want to disallow two dynamic pages - activity.php and profile.php

    If I put it into robots.txt to disallow /profile.php - will that disallow all the wildcards like /profile.php?mode=viewprofile&u=92

    if not, can this be done in htaccess?
     
    mdvaldosta, Dec 1, 2005 IP
  2. mdvaldosta

    mdvaldosta Peon

    Messages:
    4,079
    Likes Received:
    362
    Best Answers:
    0
    Trophy Points:
    0
    #2
    bump in the night
     
    mdvaldosta, Dec 2, 2005 IP
  3. Integrov

    Integrov Guest

    Messages:
    79
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #3
    maybe you can a noindex meta tag...

    or you can put the file in a diffrent directory and disallowing the directory
     
    Integrov, Jan 1, 2006 IP