1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

is this robot.txt good enough?

Discussion in 'robots.txt' started by wijasenna, Oct 25, 2011.

0
  1. #1
    i came up with this?

    is that good enough for my blog? should i add more?
     
    wijasenna, Oct 25, 2011 IP
  2. saipraveen

    saipraveen Peon

    Messages:
    36
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Need to include more Mr. wijasenna...Check it out

     
    Last edited: Oct 27, 2011
    saipraveen, Oct 27, 2011 IP
  3. WPC

    WPC Peon

    Messages:
    116
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #3
    You can extend it even more, for example. If you have low bandwidth web hosting, you don't want google to index your images. So it's a good idea to disallow access to your images folder, your css, themes, etc.
     
    WPC, Oct 27, 2011 IP
  4. qclinks

    qclinks Peon

    Messages:
    35
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #4
    That is a good suggestion indeed. I did not thing of that before. Unless of course, your images are important to be indexed.
     
    qclinks, Oct 28, 2011 IP
  5. tripbuilder

    tripbuilder Peon

    Messages:
    72
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    yes its perfect use disallow and if u add / then google will not crawl that page and u can use it like this if you want to crawl all pages..
    User-agent: *
    Disallow:
     
    tripbuilder, Nov 9, 2011 IP
  6. Iestyn

    Iestyn Member

    Messages:
    26
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    36
    #6
    Basically you want to disallow on all pages but the ones that product good content. For example on my community I've disallowed pages like profiles etc. all pages with useless information on.
     
    Iestyn, Nov 20, 2011 IP
  7. Iweber

    Iweber Peon

    Messages:
    61
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    whay not just allow the robots to everything? I say hide nothing from the G
     
    Iweber, Nov 24, 2011 IP
  8. xDiglett

    xDiglett Greenhorn

    Messages:
    2
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    11
    #8
    Because of tweaking, pageranks, bandwith usages.
     
    xDiglett, Dec 16, 2011 IP