1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

How to create robots.txt file

Discussion in 'robots.txt' started by sarahashlin, Nov 28, 2011.

0
Page 2 of 2
  1. SPA assurance

    SPA assurance Peon

    Messages:
    93
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #21
    Thanks for sharing best information about robots.txt file. It is best information for me. I like it.
     
    SPA assurance, Feb 14, 2012 IP
  2. bdthanh

    bdthanh Peon

    Messages:
    28
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #22
    You can use Google Webmaster Tools to create robots file.
    After Webmaster tools create your robots file, you can upload to your web directory
     
    bdthanh, Mar 4, 2012 IP
  3. madison37

    madison37 Greenhorn

    Messages:
    98
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    16
    #23
    1)
    User-agent: * Disallow: /
    2)
    User-agent: Googlebot-Image Disallow: /
    3)
    User-agent: * Disallow: /cgi-bin/ Disallow: /privatedir/ Disallow: /tutorials/blank.htm
    4)
    User-agent: * Disallow: / User-agent: Googlebot Disallow: /cgi-bin/ Disallow: /privatedir/
    5)
    User-agent: * Disallow: / User-agent: ia_archiver Disallow:
    6)
    User-agent: * Disallow: / User-agent: Googlebot Allow: /
     
    madison37, Mar 6, 2012 IP
Page 2 of 2