Want to dosallow URL containning '?' in particular folder

Discussion in 'robots.txt' started by TechMaster, Dec 19, 2009.

  1. #1
    Want to diasallow root-domain url which contains '?' but want to allow for a folder inside root-domain.

    Robots.txt files looks like this....

    User-agent: *
    Disallow: /imgs/
    Disallow: /images/
    Allow: /info/*?

    User-agent: Googlebot
    Disallow: /post.*
    Disallow: /donate.*
    Disallow: /*?
     
    TechMaster, Dec 19, 2009 IP
  2. xmart

    xmart Peon

    Messages:
    41
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #2
    I suggest you to generate your robot.txt file from Goggle Webmaster http://www.google.com/webmasters/ tools where it will give you the valid robot.txt file and you can verify whether it is recognized my Google bot or not.
     
    xmart, Dec 20, 2009 IP
  3. sevnrock

    sevnrock Peon

    Messages:
    241
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Goto Google Webmaster Tool > Site configuration > Settings and at the bottom you'll see Parameter handling. Google prefers you configure your parameter rules there instead of in the robots. It also lists parameters that Google found for you.
     
    sevnrock, Dec 23, 2009 IP