Dear Experts, I have a wordpress blog. I think that google is not indexing it because today I was checking it through google's webmasters tools. The was a message like "URLs submitted = 164, indexed = 1". Also there was following message in description, "Sitemap contains urls which are blocked by robots.txt.". Can someone please tell me where is the problem with my site. For reference, you may check my site from signature. Thanks
you want to edit your robots.txt. you can give the permissions to your webpages via robots.txt. i think your robot.txt file block Search crawler to index your web pages. This is a simple code on robot.txt. replace this code on your robot.txt
May be myadsbook.com solved your problem or if you need more help please suggest your website details and i will check it for further.!
Hello Friends When you feel that your website is not indexing by Google then follow the below steps to cross check it. 1. Go to Google.com enter "site: yourdomainname.com" check how many results it will display. If it shows all the pages from your website then nothing to worry. 2. If it does not list all pages of your website in Google.com then open your robot.txt and remove all content and then save and wait for next crawling. Now check again it will solve your problem. Try it.
I totally agree with that point.. Dear..Did you submit your sitemaps in your wordpress blog properly and also check the quality of content in your website.
Did you guys miss the bit where he said it was a WORDPRESS blog? Wordpress generates the robots.txt file automatically. Go take a look in the settings and ensure that it is set to allow search engines to index the blog.
Robot.txt is which where you set which of your site and link not to crawl, so make sure that your site you wish to index are not in the list of robot txt.
yep robots.txt in wordpress is automatically generated , maybe problem is when you generated sitemaps you includes url that restricted by robots.txt cmiiw
i think you need a manual submit every page in google because your robot.txt file not properly working