hello I wanted to not index some pages of my site Like http://mysite.com/page.html But http://mysite.com should index how can I do it??
Save a notepad file with the name of robots.txt filling below details in that. User-agent: * Allow: / Disallow: /
but this will also stop the index page to index in google I dont want my domain to not index in google
Then you need to disallow the page that you don't want to be index. User-agent: * Disallow: /page_name.html - Dynashox -
Put two headers , header.php and header-noindex.php , and include the file you want to in every page. Are you using a CMS ?
In that case, you'll have to contact your developer, and ask them to add that functionality for you. Once you create the page, there will be a checkbox that says "List Page on Search Engines?" That shouldn't be much of a work for them.