Main use of robots.txt in sitemap!!!!!!!!!.

Discussion in 'Google Sitemaps' started by singh03, Jul 1, 2009.

0
  1. #1
    Someone suggested, What is the Main use of robots.txt in sitemap!!!!!!!!!.:confused:
     
    singh03, Jul 1, 2009 IP
  2. DC Recruiters

    DC Recruiters Peon

    Messages:
    47
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    You use robots.txt to tell search engine spiders which sections or which types of content you'd like indexed and which you'd like them to ignore.

    These types of commands can be issued manually on each page of your site with HTML, however this makes the HTML source code of your pages longer. The problem with longer source code is that some SEOs believe that the search engines favor lean efficient code and penalize sites that have an unfavorable code to content ratio.
     
    DC Recruiters, Jul 4, 2009 IP
  3. websitetools

    websitetools Well-Known Member

    Messages:
    1,513
    Likes Received:
    25
    Best Answers:
    4
    Trophy Points:
    170
    #3
    websitetools, Jul 9, 2009 IP
  4. mpreyesmr

    mpreyesmr Banned

    Messages:
    175
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Robots.txt allow for spiders to tell the search engines what should be indexed and what shouldn't be indexed.
     
    mpreyesmr, Jul 26, 2009 IP
  5. Robertnelsoninc

    Robertnelsoninc Peon

    Messages:
    37
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #5
    You have to be able to tell the spiders exactly what you want them to do.
     
    Robertnelsoninc, Aug 4, 2009 IP