Hi guys I've been trying to get Google to update its listings for my site but it won't. Been trying for weeks and weeks! Here is the robots.txt file I am using: User-agent: * Disallow: /private.html Is this causing a problem? I am trying to block only private.html from being listing, not the whole site! It's really frustrating!!!
Create a sitemap and add the following line to your robots.txt file. Sitemap: http:// yourdomain dot com / sitemap.xml (without the spaces) Also, make sure in your index.php or index.html you have the following line. <meta name="robots" content="follow" />
You have added correct line. The suggestion you have received is ok. I can write you the name of a website where you can verify your website. It is http://www.robotstxt.org/