How to check robots.txt on site?

You can also check in Woorank or Chrome SEO to check a specific website's robots file.

 

Hi

Hey buddy just www.domainname.com/robots.txt

 

Just type domain name/robots.txt and enter file will be shown

 

urlofthesite/robots.txt
ex- http://www.abc.com/robots.txt

 

first type your domainname then type robots.txt after domainname like - www.domainname.com/robots.txt

 

Thanks for taking the time to discuss this, I feel strongly about it and love learning more on this topic.

 

By webmastertool you can check your robot.txt file....

 

Thanks for sharing this valuable post,it such a very informative.

 

You can find or create robot.txt at webmaster tool also.

 

Its easy, you can check by writing robots.txt after your domain name. For Example- http://www.domainname.com/robots.txt
Also, you can check on google webmaster tool > select your website > Crawl > robots.txt Tester.

Hope this will help you.

 

Thanks i checked as per this post http://www.jaibiz2day.com
result
User-agent: *
Disallow: /feeds
Allow: /_/rsrc/
Disallow: /_/
Sitemap: http://www.jaibiz2day.com:80/system/feeds/sitemap

travel trends

 

you can simple type "example.com/robots.txt" if it's added it will appear or it will redirect homepage if server didn't found it's very easy to add there but it's a important file you can give the instruction to google that which pages or modules you don't want crawl on search engine.

 

You can check the robots.txt by using "websitedomain/robots.txt"

 

just type the domain name yourdomain.com/robots.txt you will see if file is added if not you can add in root directory of your domain