As you can see in image that 2377 urls are blocked by robots.txt in webmastertool. So how can i fid those blocked url and unblock them . Plz help me regarding this .. My visitors dropped to 400 from 1000....my high impression page that is http://www.fansofapk.com/search/label/Android themes is bocked and its always showing in webmaster tools in health section...So i want every thing should get index and plz tell me which robots.txt file will be correct for my site... Here is my robots.txt http://www.fansofapk.com/robots.txt
Hello Regarding robots.txt there is a simple code of just two lines - User agent: Disallow: There is no such any other code which relates to robots. Make sure there is no allow term in robots.txt file. User agent self allows all crawler to crawl your website and disallow term disallow the particular page, folder and particular webpage which you doesnt want to crawl . Suppose a particular webpage is under construction and you doesn't want to crawl that particular page then you put that page in disallow term.