Doctype and Robots Text

[steve_gts]

Hi All,

Just a simple one probably!

Should I have my Doctype on every page in the site or just the index page?

And same question for Robots text?

Thanks

[Jean-Luc]

Every HTML page should have its own doctype.

robots.txt is a simple ASCII text file (no doctype).

Jean-Luc

[steve_gts]

Thanks, Ill get busy adding the doctype, but........................

robots.txt is a simple ASCII text file (no doctype).

I dont understand what you mean, someone did a robots txt for me:

<meta name="ROBOTS" content="index, follow">
<meta name="ROBOTS" content="ALL">

Should this be on every page and in the same format?

[Jean-Luc]

robots.txt is an optional file that can restrict access to the site to web robots.

The robots META tag you are using is something else :
<meta name="robots" content="index,follow">has exactly the same meaning as<meta name="robots" content="all">They say that visiting search engine robots are invited to index the page and to follow the links.

Jean-Luc

[steve_gts]

I see!

So I should add <meta name="robots" content="all"> to every page and remove <meta name="robots" content="index,follow"> from the index as it's basically duplication of ="all" ?

So i guess I don't need a robots text as well as this then ?

[steve_gts]

Aaaaaand................

Should I also add both the Doctype and Robots meta on includes pages ?

[Jean-Luc]

You do not need robots.txt and you do not need the robots META tags.

In the early search engines days, a few very polite robots did not index sites that did not give an explicit perrmission. Today all web robots consider that they may index your site except if you instruct them not to do it.

Jean-Luc

[steve_gts]

Thanks for that!

[TechnoGeek]

Very well explained by Jean-Luc. I will only try to round up the things a little. The file 'robots.txt' is a normal text file that resides in the home directory of the site. It is normally used only when you want to restrict the access to some parts of the site. If that is not your case, you could use it nevertheless if you want to have less 404 (page not found) codes.
The meta tags referring to robot behavior follow the same rules. You must include them in every page that you don't want to be indexed or whose links you don't want to be followed. If you do not include a meta tag with NOINDEX or NOFOLLOW, the page will be indexed and the links in it will be followed.
The DOCTYPE line, on the other hand, should be included in all pages.