Difference between /category/ and /category disallowing ?

Discussion in 'robots.txt' started by crystak, Dec 29, 2007.

0
  1. #1
    The example in the title - what happens if I disallow /category or /category/ ?

    Is it if you want for example the following:

    domain.com/category page not to get indexed but want all the other

    domain.com/category/news pages to be indexed for example ?

    And which one does what ?
     
    crystak, Dec 29, 2007 IP
  2. just-4-teens

    just-4-teens Peon

    Messages:
    3,967
    Likes Received:
    168
    Best Answers:
    0
    Trophy Points:
    0
    #2
    i dont think it makes a difference both would disallow category
     
    just-4-teens, Dec 29, 2007 IP
  3. crystak

    crystak Well-Known Member

    Messages:
    1,121
    Likes Received:
    36
    Best Answers:
    0
    Trophy Points:
    115
    #3
    So is there no way I can stop the /category archive from getting indexed... but all my other posts that belong to that category e.g. /category/post to be indexed ?
     
    crystak, Jan 1, 2008 IP
  4. SwapsRulez

    SwapsRulez Peon

    Messages:
    32
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Then you have to mention all the files & folders inside them in allow tag. But i think this will create the ambiguity for the bot & bot will destroy your site.. hehe... Just joking.. :p
     
    SwapsRulez, Jan 12, 2008 IP