How can i avoid the indexing of forum pages but allow thread pages?

Discussion in 'robots.txt' started by serkom, May 10, 2009.

  1. #1
    serkom, May 10, 2009 IP
  2. okili

    okili Peon

    Messages:
    2
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    This is exactly the question I have.

    Its simple to disallow any file or directory, such as:
    disallow: /forums/
    disallow: /forums/somefile.php

    can it allow or disallow urls using some predefined pattern?

    I mean, I have some urls like this:
    /forums/index.php?act=Post&CODE=02&f=15&t=13&qpid=15
    /forums/index.php?act=Post&CODE=02&f=15&t=13&qpid=85
    /forums/index.php?act=Post&CODE=02&f=15&t=14&qpid=16

    and I do not want bots to index them, and I want to disallow the urls, which start on:
    /forums/index.php?act=Post&CODE=02 ?

    if I write the following in robots.txt
    disallow: /forums/index.php?act=Post&CODE=02
    or
    disallow: /forums/index.php?act=Post&CODE=02*

    will it work? or it will disallow the whole /forums/ directory or block /forums/index.php file totally?

    I do not want to test it till I'm not sure, while google has indexed more than 35000 urls from my webpage and there is a big threat in modifying robots.txt.
     
    okili, May 16, 2009 IP
  3. linkdealer

    linkdealer Active Member

    Messages:
    138
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    90
    #3
    Simply disallow: /forumdisplay.php?f=*
     
    linkdealer, Jun 9, 2009 IP