What Does Robots=all do for Search Crawlers?

Discussion in 'Google' started by Ascendancy, May 26, 2010.

  1. #1
    I have been checking a lot of blogs recently and seen the meta tag value for Robots like this:

    <meta name="Robots" content="All" />
    
    Code (markup):
    I'm wondering how does this affect crawlers? Is this the same as placing every link on the page as DoFollow?
     
    Ascendancy, May 26, 2010 IP
  2. mdamin76

    mdamin76 Well-Known Member

    Messages:
    1,752
    Likes Received:
    146
    Best Answers:
    0
    Trophy Points:
    153
    #2
    That means you allow all search engines to crawl your site.
     
    mdamin76, May 26, 2010 IP
  3. Sxperm

    Sxperm Notable Member

    Messages:
    4,386
    Likes Received:
    142
    Best Answers:
    0
    Trophy Points:
    225
    #3
    It is an outdated meta tags. It was used to tell Googlebots to crawl on contents but not useful for now. Many of website has been crawled without put this tag in use. You can read further on this discussion - <meta name="robots" content="ALL">
     
    Sxperm, May 26, 2010 IP
  4. Ascendancy

    Ascendancy Active Member

    Messages:
    1,721
    Likes Received:
    31
    Best Answers:
    0
    Trophy Points:
    90
    #4
    Oh okay yeah that link was informative. Then what is the meta tag for setting all links on the page to DoFollow? Are they set to DoFollow by default without any meta tag settings?
     
    Ascendancy, May 26, 2010 IP
  5. SeoKungFu

    SeoKungFu Active Member

    Messages:
    206
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    55
    #5
    Naw, it means nothing. Nothing at all - it's a nonsense. Robots are made to follow, index and archive pages and the ONLY meaningful clauses in the meta tags section are those that restrict them.
     
    SeoKungFu, May 26, 2010 IP
  6. reapr

    reapr Peon

    Messages:
    1,711
    Likes Received:
    18
    Best Answers:
    0
    Trophy Points:
    0
    #6
    Worthless nowadays. It use to get honored by most bots back in the day.
     
    reapr, May 26, 2010 IP
  7. bermuda

    bermuda Peon

    Messages:
    868
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #7
    When you do not add any robot file to your web server, that means all types of bots and search engines crawlers are allowed to crawl all sections and folders of your website. This is recommended in most cases because naturally a lot of webmasters like to find most of their pages indexed on the net.
     
    bermuda, May 26, 2010 IP
  8. DoDo Me

    DoDo Me Peon

    Messages:
    2,257
    Likes Received:
    27
    Best Answers:
    0
    Trophy Points:
    0
    #8
    you can simple remove it and means the same thing
     
    DoDo Me, May 26, 2010 IP
  9. Michael smith

    Michael smith Peon

    Messages:
    313
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #9
    it means u allow all the search engine to crawl your site, but this is old one no one use this code in meta tag.
     
    Michael smith, May 27, 2010 IP
  10. rafderamas

    rafderamas Active Member

    Messages:
    288
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    53
    #10
    It means that you allow search engines to crawl everything on your site.
     
    rafderamas, May 27, 2010 IP
  11. yaseen4u

    yaseen4u Peon

    Messages:
    461
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #11
    yes i agree with you.....
     
    yaseen4u, May 27, 2010 IP
  12. marketingguru101

    marketingguru101 Active Member

    Messages:
    536
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    73
    #12
    thanks for sharing this information.
     
    marketingguru101, May 27, 2010 IP
  13. buzinessware

    buzinessware Peon

    Messages:
    408
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #13
    "All" is you are allow all search engines to crawl your site.
     
    buzinessware, May 27, 2010 IP