How to stop my all external link crawling?

Discussion in 'robots.txt' started by intellg, May 5, 2010.

0
  1. #1
    As title says I want to stop crawling all my external link by rovots.txt. Is there any formula available in robots.txt ? If yes then please share with me here.
     
    intellg, May 5, 2010 IP
  2. onlinesujeet

    onlinesujeet Greenhorn

    Messages:
    49
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    16
    #2
    add this code external pages and any search engine crawling stop.
    <head>

    User-agent: *
    Disallow: /

    </head>
     
    onlinesujeet, May 18, 2010 IP