1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Using robots.txt to disallow to spider dynamic URL

Discussion in 'robots.txt' started by MoneyTopList.com, Oct 21, 2009.

0
  1. #1
    If is use in my robots.txt this:

    User-agent: *
    Disallow: info.php

    ...will I disallow robots to spider also dynamic URLs like mydomain/info.php?id=something-dynamic?
     
    MoneyTopList.com, Oct 21, 2009 IP
  2. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,682
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
    #2
    If you use this robots.txt code, it will block only one page that would be info.php. It won't block any other dynamic page.
     
    manish.chauhan, Oct 28, 2009 IP
  3. MoneyTopList.com

    MoneyTopList.com Active Member

    Messages:
    314
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    88
    #3
    OK, thank you for your reply. Is there any other solution how to block the dynamic URLs?
     
    MoneyTopList.com, Oct 28, 2009 IP
  4. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,682
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
    #4
    Yes, you can simply block those urls like
    User-agent: *
    Disallow: /info.php
    Disallow: /info.php?id=*
     
    manish.chauhan, Oct 28, 2009 IP
    digitalpoint and MoneyTopList.com like this.
  5. yesrams

    yesrams Greenhorn

    Messages:
    10
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    11
    #5
    Good answer
     
    yesrams, Oct 28, 2009 IP