1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Help Understanding Robots.txt basic commands

Discussion in 'robots.txt' started by sarahturned, Dec 30, 2010.

  1. #1
    Hi,

    I'd like to clean up some of the duplicate content on some of my blogs.

    What would be the difference between these three commands in robots.txt

    Disallow: /tag/

    and

    Disallow: /tag/*

    and

    Disallow: */tag/

    Thank you
     
    sarahturned, Dec 30, 2010 IP
  2. fear

    fear Notable Member

    Messages:
    3,747
    Likes Received:
    222
    Best Answers:
    0
    Trophy Points:
    205
    #2
    fear, Jan 2, 2011 IP
  3. cipals15

    cipals15 Well-Known Member

    Messages:
    1,086
    Likes Received:
    9
    Best Answers:
    0
    Trophy Points:
    100
    #3
    cipals15, Jan 2, 2011 IP
  4. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,684
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
    #4
    As Cipals said, /tag/ and /tag/* work the same way, they both restrict all urls starting from tag directory. This command Disallow: /*/tag/ will be used to block all urls that do have tag at the second level on their urls like http://yourdomain.com/abc/tag/pqr.html etc
     
    manish.chauhan, Jan 3, 2011 IP
  5. daeng palopo

    daeng palopo Member

    Messages:
    312
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    30
    #5
    Try using plugin robot.txt.. :)
     
    daeng palopo, Apr 19, 2011 IP
  6. brad.smith4321

    brad.smith4321 Peon

    Messages:
    249
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #6
    its tag of robots.txt for show no follow link
     
    brad.smith4321, Apr 19, 2011 IP