i want robots.txt for yahoo, google and msn

Discussion in 'Google' started by yaseen4u, May 10, 2009.

0
  1. #1
    i want robots.txt for yahoo google and msn for indexing my all page in search engine
    pls help me..............
     
    yaseen4u, May 10, 2009 IP
  2. Abhik

    Abhik ..:: The ONE ::..

    Messages:
    11,337
    Likes Received:
    606
    Best Answers:
    0
    Trophy Points:
    410
    Digital Goods:
    2
    #2 
    User-agent: Google
Disallow:

User-agent: Yahoo! Slurp
Disallow:

User-agent: MSNBot
Disallow:

User-agent: *
Disallow: /
    Code (markup):
    But, that won't help in indexing.. You need to submit sitemaps to the Search Engines to get your pages crawled and Indexed.
     
    Abhik, May 11, 2009 IP
  3. .htaccess

    .htaccess Peon

    Messages:
    277
    Likes Received:
    40
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Hey, Your last statement about * will not help the site to index. !!
    It can be like this

    
User-agent: *
allow: /
    Code (markup):
    No need to add everything, just add lines specified by me. and submit your site to various search engines manually.
     
    .htaccess, May 11, 2009 IP