Which is better Robots.txt or Meta Robots Tag?

Both are importance for a website.

 

Yes i agree with danish001 we can disallow the files In robots.txt which file we want to avoid from crawler.
Well i have first research on Meta robots tags and then i will tell you.

 

Robot.txt is a text file and meta robot is HTML file. Robot.txt.It page level instruction.It can stop search engines to fetch the restricted contents but on the other side and to obey the meta robot tag first they need to fetch the contents at least and this will be counted towards bandwidth usage.If your web host prohibits you from uploading "robots.txt" to the root directory, or you simply wish to restrict crawlers from a few select pages on your site, an alternative to "robots.txt" is to use the robots meta tag.

 

Hi Mikenielson,

Can you help me on some basics question of seo i am confused.
sitemap.xml, how to analysis a fresh website ?

 

I think Both are best for Web Site...........

 

Robots.txt can prevents (or allow) bots from accessing certain files/pages/directories.
Robots Meta Tag let you to control whether to allow indexing of a certain page, or following outgoing links from that page.

 

Every one is saying "both are best"..Instead you can give a thumb for first right answer.
Sorry friends (if hurted) but it is good way to participate in forum.

 

Both of them are used for crawling purpose If you block a page with robots.txt, Google bot will never crawl the page and will never read any meta tags on the page.If you allow a page with robots.txt but block it from being indexed using a meta tag, Google bot will access the page, read the meta tag, and subsequently not index it.
1. Robots.txt won't stop a URL being displayed in Google serps.
2. meta 'noindex' will stop a URL being displayed in Google serps.

 

You should use both for best results.

 

Well i have first research on Meta robots tags and then i will tell you.

 

Robot.txt contains information for the website. You can see the site by "birlamedisoft.com/robots.txt "

 

Robots.txt and Meta Robots Tag are both important for well better website , Thanx every one.

 

I cannot ignore both robot txt and robot meta tag.
You can use a special HTML <META> tag to tell robots not to index the content of a page, and/or not scan it for links to follow.
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">