How i can block Dynamic URLs Through Robots.txt

Discussion in 'Google' started by yogeshseo, Jul 25, 2011.

0
  1. #1
    I have Dynamic website and i wanna block some dynamic urls through robots.txt can you tell me how i can do this??
     
    yogeshseo, Jul 25, 2011 IP
  2. danielwood

    danielwood Member

    Messages:
    115
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    28
    #2
    Hi, to block all dynamically generated URLs, add the following code in your robots.txt:

    user-agent: *
    disallow: /filename.php?

    this would block all files having url like www.domain.com/filename.php?cid=21
     
    danielwood, Jul 25, 2011 IP
  3. yogeshseo

    yogeshseo Active Member

    Messages:
    103
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    51
    #3
    how i can 301 redirection on these urls?
     
    yogeshseo, Jul 25, 2011 IP
  4. yogeshseo

    yogeshseo Active Member

    Messages:
    103
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    51
    #4
    yogeshseo, Jul 25, 2011 IP
  5. danielwood

    danielwood Member

    Messages:
    115
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    28
    #5
    could you explain what problem are you getting?
     
    danielwood, Jul 25, 2011 IP
  6. danielwood

    danielwood Member

    Messages:
    115
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    28
    #6
    Do you want redirection code or looking for URL rewriting?
     
    danielwood, Jul 25, 2011 IP
  7. MRFD

    MRFD Active Member

    Messages:
    31
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    93
    #7
    For redirections you can use an .htaccess file.
     
    MRFD, Jul 25, 2011 IP
  8. yogeshseo

    yogeshseo Active Member

    Messages:
    103
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    51
    #8
    i m using .htaccess file but its not working properly.
     
    yogeshseo, Jul 26, 2011 IP
  9. MRFD

    MRFD Active Member

    Messages:
    31
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    93
    #9
    Post your .htaccess file here so we could check what's wrong...
     
    MRFD, Jul 27, 2011 IP
  10. bhavatmaj

    bhavatmaj Well-Known Member

    Messages:
    319
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    103
    #10
    remove this line from robots.txt

    Disallow: /.html

    Disallow: /http://www.indiatodaybazaar.com
     
    bhavatmaj, Jul 27, 2011 IP