1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

What this robot.txt will do ? Please help !!

Discussion in 'robots.txt' started by Rajnish357, Jan 23, 2012.

0
  1. #1
    Please Help me understand this content of robot.txt file

    User-agent: *
    Allow: /
    Disallow: /category/education/
    Disallow: /category/jobs/
    Disallow: /category/business/

    I want to know what this will do ??


    I have a WordPress Blog and I want to all robots on first page of category http://www.digmlm.com/category/education but not on /page/2, /page/3, /page/4 and So on

    But I think I will use above code in robot.txt file it will all stop indexing article !!!
     
    Rajnish357, Jan 23, 2012 IP
  2. tiffanywilliams12i2

    tiffanywilliams12i2 Peon

    Messages:
    164
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    yes the abode would stop indexing
     
    tiffanywilliams12i2, Apr 10, 2012 IP
  3. carlhenson

    carlhenson Peon

    Messages:
    45
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    You can use:

    Without the trailing slashes, I think that will do what you want.
     
    carlhenson, Apr 11, 2012 IP