Robots .txt restricted - assistance needed

Discussion in 'robots.txt' started by snooks, Nov 9, 2007.

0
  1. #1
    Hi guys.

    I have a problem with robots.txt being restricted, i did get an answer from somebody else but dont fully understand it as im new and still learning.

    He replied:
    product.php can be disallowed because you use CDSEO for categories and products.

    This line of your robots.txt is preventing the dynamic urls from being indexed, including pages.php?pageid=

    Disallow: /*?

    If you want to allow the pages.php?pageid= links then add this code directly after that on the next line:

    Allow: pages.php

    Can somebody just rewrite, or advise exactly what i have to change. I cant afford to make a mistake so very simple, just put:


    add this ******************

    remove this ******************

    Then i cant mix it up.

    Many thanks
     
    snooks, Nov 9, 2007 IP
  2. Kuldeep1952

    Kuldeep1952 Active Member

    Messages:
    290
    Likes Received:
    18
    Best Answers:
    0
    Trophy Points:
    60
    #2
    Can you please post the contents of your robots.txt here.
    Then someone will be able to help you better.
     
    Kuldeep1952, Nov 9, 2007 IP
  3. snooks

    snooks Well-Known Member

    Messages:
    1,054
    Likes Received:
    98
    Best Answers:
    0
    Trophy Points:
    175
    #3
    Thank you....i was just about to do that but recieved a pm with the answer.

    Thanks guys.
     
    snooks, Nov 9, 2007 IP