Hi DPers, I have some pages on my website that I dont want google to index. I shouldn't mention these urls in the sitemap at all or should I mention these urls in the sitemap and block access to these urls via the robots.txt file? What would be a better solution for my site? Thanks Al
Won't that be hiding information from google? Isn't it a better idea to add the pages in the sitemapand block it via robot.txt altogether? Al
Don't mention them in your sitemap. But even if you mention, you can restrict them in robots.txt. It has higher priority. You can restrict a single page Disallow: /links.html or a directory Disallow: /admin/