Please give me the solution how can i solved this problem. I have no robot text file in the main directory. But if i click the browser and click my site urls http://cricketalways.com/robots.txt then the result comes is User-agent: * Disallow: /wp-admin/ Disallow: /wp-includes/ Sitemap: http://cricketalways.com/sitemap.xml.gz its my wordpress site. but my forum blocked by robots.txt see this image my webmaster tools sitemap Please give me the solution. will i create robots.txt file and upload it public html? If i have to create what format should be. Please expertise comments need.
Hi there, it was showing in my google webmaster tool as well. do one thing in google webmaster tool ,their is way of adding the new page(in health folder i think so). why do not add their you pages and they will get index. so do not worry. it will really work..
use "fetch as google bot" feature in webmaster tools for your forum URL. If it is blocked, it will tell you by what rule and what version of your robots.txt googlebot is using while asuming it is not allowed to access it