This one is for robots.txt expertsL How does one pull his off...

Discussion in 'robots.txt' started by astounding, Jul 18, 2007.

  1. #1
    Here's the drill:

    My forum posts are SEO friendly and in .html format.

    So typical post link:
    http: //www.site.com/forum/how-to-cut-your-head-off.html
    http: //www.site.com/forum/owning-counter-strike.html

    Now what would I enter into robots.txt in order to prevent from SE's from indexing:

    http: //www.site.com/forum/owning-counter-strike.html;msg122185;topicseen#msg122185

    OR

    http: //www.site.com/forum/owning-counter-strike.html;msg122197

    OR

    http: //www.site.com/forum/owning-counter-strike.html;msg122197#msg122197

    Because they all point to same link, hence I'll get penalized for duplicate content.

    Thank you.
     
    astounding, Jul 18, 2007 IP
  2. WebGeek182

    WebGeek182 Active Member

    Messages:
    510
    Likes Received:
    28
    Best Answers:
    0
    Trophy Points:
    95
    #2
    This should do it:

    
    User-agent: *
    Disallow: *.html;msg*
    
    Code (markup):
     
    WebGeek182, Jul 18, 2007 IP
  3. astounding

    astounding Well-Known Member

    Messages:
    167
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    103
    #3
    Thanks.

    I had:

    Disallow: /forum/*.msg*
    Disallow: /forum/*msg*

    I now commented them out, and put yours (Disallow: /forum/*.html;msg*) instead.
     
    astounding, Jul 18, 2007 IP
  4. trichnosis

    trichnosis Prominent Member

    Messages:
    13,785
    Likes Received:
    333
    Best Answers:
    0
    Trophy Points:
    300
    #4
    add the code which is above will solve your problem
     
    trichnosis, Jul 22, 2007 IP