Is it possible to have */

Discussion in 'robots.txt' started by murph, Sep 14, 2006.

  1. #1
    I need to block certain "directories" that appear after lots of different main directories. (I say directories, but in fact it is just mod_rewrite used to create nice urls!) Can i do this..eg:

    User-agent: *
    Disallow: */rate/

    to block all these eg:

    /foldera/rate/
    /folderb/rate/
    /folderc/rate/


    Or would i have to specify each and every one of them with a full path?
     
    murph, Sep 14, 2006 IP
  2. zerohex

    zerohex Peon

    Messages:
    19
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Try this site: tool.motoricerca.info/robots-checker.phtml

    zerohex
     
    zerohex, Oct 3, 2006 IP