1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

What is the robots.txt?

Discussion in 'Search Engine Optimization' started by aaateleshopping, Feb 27, 2012.

0
Page 2 of 2
  1. scanningservices

    scanningservices Peon

    Messages:
    19
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #21
    Hi..robot.txt is a file that is included in a website in order to not find the selected pages in the Google.
     
    scanningservices, Feb 29, 2012 IP
  2. mark joshef

    mark joshef Greenhorn

    Messages:
    73
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    16
    #22
    "A robots.txt file on a website will function as a request that specified robots ignore specified files or directories when crawling a site."
     
    mark joshef, Feb 29, 2012 IP
  3. jhonden

    jhonden Peon

    Messages:
    87
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #23
    Robot.txt is a process by which we hide our webpages from the search engine spiders. When we don't want to get any page craweled by spider then we use robot.txt.
     
    jhonden, Feb 29, 2012 IP
  4. shashank90

    shashank90 Peon

    Messages:
    69
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #24
    Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit.
     
    shashank90, Mar 4, 2012 IP
  5. jitendrashukla

    jitendrashukla Member

    Messages:
    194
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    26
    #25
    Nice Explanation, Thanks for sharing with us...
     
    jitendrashukla, Mar 5, 2012 IP
  6. SeoInVOgue

    SeoInVOgue Peon

    Messages:
    32
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #26
    Robot.txt is a text file webmasters create to instructs robots on how to crawl and index pages on their websites.
     
    SeoInVOgue, Mar 5, 2012 IP
  7. shashank90

    shashank90 Peon

    Messages:
    69
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #27
    [FONT= ]Robots.txt is a text file that we put on a site to tell search robots which pages you would like them not to visit.[/FONT]
     
    shashank90, Mar 20, 2012 IP
  8. jullieroslin

    jullieroslin Member

    Messages:
    74
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    43
    #28
    The robots.txt file is used to provide instructions about the Web site to Web robots and spiders. Web authors can use robots.txt to keep cooperating Web robots from accessing all or parts of a Web site that you want to keep private.
     
    jullieroslin, Mar 20, 2012 IP
  9. kalpitvthakkar

    kalpitvthakkar Greenhorn

    Messages:
    58
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    18
    #29
    it describes what pages to crawl or not to crawl by crawlers..suppose we don't want to crawl admin login page, privacy policy page or some other static pages that are not import from the point of view of SEO then you can specify those pages in to robots.txt file which tells crawlers to not to crawl those pages.
     
    kalpitvthakkar, Mar 21, 2012 IP
Page 2 of 2