Social Buttons
  1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

what is Robot's .text??

Discussion in 'Search Engine Optimization' started by shivprakash, Apr 27, 2011.

0
  1. #1
    Describe.....
    SEMrush
     
    shivprakash, Apr 27, 2011 IP
    SEMrush
  2. techendeavour

    techendeavour Peon

    Messages:
    407
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Robots.txt file is used to tell search engines not to crawl particular pages on the website.
     
    techendeavour, Apr 27, 2011 IP
  3. Vineet Koushik

    Vineet Koushik Member

    Messages:
    60
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    31
    #3
    A simple text file that stops Google (and other search engines that recognize the file and its commands) from crawling the site, selected pages in the site, or selected file types in the site.
     
    Vineet Koushik, Apr 27, 2011 IP
  4. Taktical

    Taktical Greenhorn

    Messages:
    29
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    18
    #4
    its really easy to build. 2 minutes max. Google a sample to see it
     
    Taktical, Apr 27, 2011 IP
  5. dkrt111

    dkrt111 Peon

    Messages:
    23
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Robots.txt is a text file you put on your web page to tell the search robots which pages you would like not to visit.
     
    dkrt111, Apr 27, 2011 IP
  6. MasterWeb

    MasterWeb Active Member

    Messages:
    179
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    53
    #6
    It's a file where you say to spiders, hey! don't index those pages!.
     
    MasterWeb, Apr 27, 2011 IP
  7. heliumcreative

    heliumcreative Peon

    Messages:
    22
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    Robot.txt is simple text file. If you don't want to crawl some of your web pages by google crawler then you can define it in the robot.txt file. you can generate robot.txt file from the google webmaster tools.
     
    heliumcreative, Apr 27, 2011 IP
  8. chestercaldwel

    chestercaldwel Peon

    Messages:
    326
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #8
    It's exactly what most of the people are telling here..
    It guides the search engine spider to not to crawl some specific pages...
     
    chestercaldwel, Apr 27, 2011 IP
  9. RidhvanPathan

    RidhvanPathan Peon

    Messages:
    34
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #9
    Robot.txt file is a way to tell search engine spider that which page to crawl and which page to not crawl, it is one type of simple notepad type text file....
     
    RidhvanPathan, Apr 27, 2011 IP
  10. michalka

    michalka Peon

    Messages:
    79
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #10
    Robot.txt file is used to restrict search engine from accessing pages in our site. Follow this link to Create Robot.txt file
     
    michalka, Apr 27, 2011 IP
  11. iphone4developers

    iphone4developers Guest

    Messages:
    12
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #11
    oh thanks michalka ... you are only one to give proper link to me ...... thanks once again
     
    iphone4developers, Apr 27, 2011 IP
  12. fernades

    fernades Member

    Messages:
    571
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    28
    #12
    This file simply tells the Search engine crawlers about some pages that is no need to crawl.
     
    fernades, Apr 27, 2011 IP
  13. Fadi4u

    Fadi4u Active Member

    Messages:
    135
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    71
    #13
    Robot.txt is a File that allows Search Engine To Crawl your site with effective manner..
     
    Fadi4u, Apr 27, 2011 IP
  14. SiberForum

    SiberForum Banned

    Messages:
    1,536
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    80
    #14
    I suppose that it is information on the robots exclusion protocol and how to develop a properly validated robots.txt file.
     
    SiberForum, Apr 27, 2011 IP
  15. JasonBrown

    JasonBrown Peon

    Messages:
    19
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #15
    Robots.text is an important text file from the SEO point of view. This file put restrictions of some web pages and tell the search-engines not to crawl those particular pages on which restrictions applied. It tells search-engines for not crawling restricted web-pages of your websites.
     
    JasonBrown, Apr 29, 2011 IP
  16. bobharley2011

    bobharley2011 Peon

    Messages:
    55
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #16
    What you are referring as Robots.text is actually written as robot.txt. It is a file that contains instructions for search engines robots or crawlers. Usually robot.txt is used to keep away crawlers from pages they shouldn't index or crawl.
     
    bobharley2011, Apr 29, 2011 IP