Wordpress CMS 404 question-Disallow Robot Indexing

Hi there,

I am using Wordpress 2.5 as my CMS.

I heard some SEO expert said that, for SEO reason, we shouldn't allow Google or other robot to index our 404 pages.

And IF somehow, Google do indexed some of our 404 error pages, it could very well be that our blog is not returning the proper 404 Not Found http status code, which could be a big problem.

Someone suggested If we don't want our 404's indexed by Google, we should then add this to our Wordpress header.php :

<?php if(is_404())echo '<meta name="robots" content="noindex" />';?>

Code (markup):

If we add another line of code, all of our pages except for 404 pages will be indexed. The code will be :

<?php if(is_404())echo '<meta name="robots" content="noindex" />
'; else echo '<meta name="robots" content="follow, all" />
'; ?>

Code (markup):

Question 1 : Are those the correct codes to add ?
Question 2 : WHICH PART in the header.php should I insert the code into ?

Thanks in advance for your help

 

Those are the correct codes it seems, and, you will have to add them just before the body section

 

Thanks for your reply. But I was being suggested (in another forum) to ignore the 2nd line of code which I've mentioned up there.

Have a nice day.

 

This is a load of garbage. If the server is returning the proper HTTP header to the search engine spiders, those pages will NOT be getting indexed to begin with.

Who gave you this baloney anyway?

 

Hi Dan, you are right, I have asked this in another forum and I have decided not to edit/hack the header.php ... Thanks anyway

 

<?php header("HTTP/1.1 404 Not Found"); ?>
<?php header("Status: 404 Not Found"); ?>

PHP:

Put that at the very top of your WordPress 404.php file - before the get_header(); call.

Here's an example of a custom 404.php file I'm using for the WordPress Theme Framework I have under development:

<?php header("HTTP/1.1 404 Not Found"); ?>
<?php header("Status: 404 Not Found"); ?>

<?php get_header(); ?>

	<div id="content">
		<div class="wrapper" id="error404">
			<h1>Oops! The page you requested is not here!</h1>
			<p>I&#8217;m sorry, but the page or file
<?php
#some variables for the script to use
#if you have some reason to change these, do.  but wordpress can handle it
$adminemail = get_bloginfo('admin_email'); #the administrator email address, according to wordpress
$website = get_bloginfo('url'); #gets your blog's url from wordpress
$websitename = get_bloginfo('name'); #sets the blog's name, according to wordpress

  if (!isset($_SERVER['HTTP_REFERER'])) {
    #politely blames the user for all the problems they caused
        echo "you were looking for at "; #starts assembling an output paragraph
	$casemessage = "could not be found.";
  } elseif (isset($_SERVER['HTTP_REFERER'])) {
    #this will help the user find what they want, and email me of a bad link
	echo "you were looking for at"; #now the message says I'm sorry, but the page or file you were looking for at...
        #setup a message to be sent to me
	$failuremess = "Someone, probably a legitimate user, tried to go to $website"
        .$_SERVER['REQUEST_URI']." and received a 404 (file not found) error. ";
	$failuremess .= "It likely wasn't the visitor's fault, so please try to fix this as soon as possible.  
        They came from ".$_SERVER['HTTP_REFERER'];
	mail($adminemail, "404 Error Notice For ".$_SERVER['REQUEST_URI'],
		$failuremess, "From: $websitename <noreply@".(str_replace("www.", "", str_replace("http://", "", $website))).">"); #email you about problem
	$casemessage = "The site administrator has been emailed about this problem and will try to fix it as soon as possible."; #set a friendly message
  }
  echo " ".$website.$_SERVER['REQUEST_URI']; ?> 
<?php echo $casemessage; ?>
			</p>
			<div>
				<h2>You may not be able to find the page or file because of:</h2>
				<ol>
					<li>An <strong>out-of-date bookmark/favorite.</strong></li>
					<li>A search engine that has an <strong>out-of-date listing for this site.</strong></li>
					<li>A <strong>mis-typed address</strong>.</li>
				</ol>
				<h2>Please try one of the following pages:</h2>
				<ul>
					<li><a href="/" title="<?php bloginfo('name'); ?>&#8217;s home page">Home Page</a></li>
					<li><a href="/sitemap/" title="<?php bloginfo('name'); ?>&#8217;s Site Map">Site Map</a></li>
				</ul>
				<p>Alternatively, you can search for what you&#8217;re looking for:</p>
				<?php include(TEMPLATEPATH . "/searchform.php"); ?>
				<small><span>&#187;</span> Error: 404 - File Not Found</small>
			</div>
		</div>
	</div>
<?php get_sidebar(); ?>

<?php get_footer(); ?>

PHP:

And here's the CSS code that goes with it:

/* 404 error page */
#error404 {
	/* any specific style rules for the 404 div go here (such as removing margins to allow for 100% width) */
}
	#error404 h1 {
		color: #DC143C;
		font: bold 1.5em/2 georgia, garamond, "times new roman", times, serif;
	}

	#error404 p {
		padding: 0 0 0 1em;
	}

	#error404 h2 {
		font: bold 1em/1.5 georgia, garamond, "times new roman", times, serif;
		margin: 0.5em 0;
	}

	#error404 ol {
		list-style-type: lower-alpha;
		margin: 0 0 0 2.4em;
	}

	#error404 ul {
		margin: 0 0 0 1.5em;
	}
		#error404 strong {
			font-style: italic;
			font-weight: normal;
		}

	#error404 small {
		color: #DC143C;
		display: block;
		font-size: 0.85em;
		font-weight: bold;
		margin: 2em 0 0 0;
	}
		#error404 small span {
			font-size: 1.25em;
		}

Code (markup):

(Bear in mind I already use a reset so there's no need to kill the margins and padding again.)