You can use Robots.txt to disallow search engine from crawling. (you need to place this robots.txt file in your root dir of website : example.com/robots.txt & content of the file is explained below) Disawllow indexing of a specific folder from all search engine bots. User-agent: * Disallow: /folder/ Disallow Googlebot from indexing of a folder, except for allowing the indexing of one file in that folder User-agent: Googlebot Disallow: /folder1/ Allow: /folder1/myfile.html You can also use the meta tags : When you block URLs from being indexed in Google via robots.txt, they may still show those pages as URL only listings in their search results. A better solution for completely blocking the index of a particular page is to use a robots noindex meta tag on a per page bases. You can tell them to not index a page, or to not index a page and to not follow outbound links by inserting either of the following code bits in the HTML head of your document that you do not want indexed. <meta name="robots" content="noindex"> <-- the page is not indexed, but links may be followed <meta name="robots" content="noindex,nofollow"> <-- the page is not indexed & the links are not followed. Please note that if you do both: block the search engines in robots.txt and via the meta tags, then the robots.txt command is the primary driver, as they may not crawl the page to see the meta tags, so the URL may still appear in the search results listed URL-only.
I am not talking about about a page or folder. I am talking about specific content on a web page. Let's sat a paragraph on a WebPage.
Use this, hope this will help you better. <meta name="robots" content="noindex,nofollow"> <-- the page is not indexed & the links are not followed.
This will deindex a whole webpage. What I asked is how I can prevent certain area say a div tag, or a paragraph of content on a webpage from indexing. Not a whole web page.
<p>This is normal (X)HTML content that will be indexed by Google.</p> <!--googleoff: index--> <p>This (X)HTML content will NOT be indexed by Google.</p> <!--googleon: index> Code (markup): Full Article Here: https://perishablepress.com/tell-google-to-not-index-certain-parts-of-your-page/