Trying to verify a Google sitemap I get an error message like this:

[miko67]

Google gives me this message and explanation when I try to verify a Sitemap:

We've detected that your 404 (file not found) error page returns a status of 200 (OK) in the header.
This configuration presents a security risk for site verification and therefore, we can't verify your site. If your web server is configured to return a status of 200 in the header of 404 pages, and we enabled you to verify your site with this configuration, others would be able to take advantage of this and verify your site as well. This would allow others to see your site statistics. To ensure that no one can take advantage of this configuration to view statistics to sites they don't own, we only verify sites that return a status of 404 in the header of 404 pages. Please modify your web server configuration to return a status of 404 in the header of 404 pages. Note that we do a HEAD request (and not a GET request) when we check for this. Once your web server is configured correctly, try to verify the site again. If your web server is configured this way and you receive this error, click Check Status again and we'll recheck your configuration.

and I have no idea what to do about it.

Is it all about mod-rewriting in my .htaccess file?

If it is about mod-rewriting, do anyone here have any idea what to write?

Incidentally Google seems to be indexing (all my pages more or less) and then de-indexing quite a lot on a daily basis... I wonder what that is all about?

[T0PS3O]

You probably have custom 404 pages, right?

Maybe you redirect 404's to a standard page? Basically, the issue is your 404s throw up a 200 status, not a 404. If you are not familiar with those things then it must have been something your host has kindly set up, but wrongly so! Contact them I'd say.

[Matts]

It's about http status codes, and what your server is returning when a page is not found.

200 OK means the file was found
404 Not Found means the file doesn't exist

These are status codes sent in the http header, and you can't see them directly. You either need to use ethereal to sniff and watch, or look in the access_log of the web server, where you'll see them logged for each request sent to the server.

So what G is doing is having you place a file on your server with a unique name, and then they access it [using HEAD and not GET] to see if it exists. The server should return a 200 Ok. Then G requests a random made up name like "test_404_response_page_ahas9a0aa01ajHasn", which should return a 404 Not Found. If the GoogleBot receives a 200 Ok then it means that the server always returns a 200 Ok, regardless of whether the file is there or not, and so G can't actually verify the file exists [i.e. the server always says the file exists no matter what].

If you have a custom 404 page that says 'sorry page not found' and it's not sending a 404, and you have php installed add to the top of the custom 404 page:
<?php
header('HTTP/1.0 404 Not Found');
?>

[miko67]

Originally Posted by Matts

If you have a custom 404 page that says 'sorry page not found' and it's not sending a 404, and you have php installed add to the top of the custom 404 page:
<?php
header('HTTP/1.0 404 Not Found');
?>

ok, thx a lot. Tried this one.

My wordpres theme does not have a 404 page, but an post loop on the index page, if no page match - then message = "Sorry, no posts".

I tried to make an include to 404.php, but that inserts the 404 page in the index (not good because now I get a header error with your php header inserted in the middle of the body).

I need a php command to move to 404.php (I'm a php novice still) - How do I do this?

[Matts]

Look at this page on your site: www.coffeecoffee.be/404_qwerty

Find out why that's showing up. It's not a Wordpress issue. Your server is configured to render that page when a page isn't found. Once you know that and fix it, the 404's should get returned.

[miko67]

Update:

I am now trying this:

<?php
$page = file_get_contents('404.php');
echo $page;
?>

and It gives me a beautiful 404 not found message with no reported errors (you can see it on: http://www.coffeecoffee.be/blahblah). However I cannot get it to verify because the header is still not what it is supposed to be.

I guess it is because I put the contents of the 404.php page (including the "<?php header("HTTP/1.1 404 Not Found"); ?>" sentence) in the body of the page, which obviously is wrong, but I cannot figure out how to clear the page so the header is being red as a header - if you understand what I mean?

It's close to midnight here, I'm deep in php books that used to be unopened, and I am not resting until this is working.

Any help is truly appreciated.

[T0PS3O]

Rather than looking in the php code I'd check the underlying issue of your server doing this. Though there is a chance that it is the application (WP) that has some application top functionalilty that creates those pages.

If you want to stick with the php fix then do a header 301 redirect to that 404 page so you'll end up eventually with the correct 404 header.

[miko67]

Originally Posted by T0PS3O

If you want to stick with the php fix then do a header 301 redirect to that 404 page so you'll end up eventually with the correct 404 header.

Allright, you're giving me more things to question here, but I'll read up on it and see if I can figure it out (a header 301 redirect ).

Meanwhile... This is my source for any non existing URI, showing how the code described earlier results in a 404.php page showing up in the main text area of the body of the index file.

I basically want to use php to get rid of all the stuff I made blue so that the header can be read again (I'm propably not making a lot of sense any more but please bear with me).

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head profile="http://gmpg.org/xfn/11">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>
Coffee Coffee</title>
<meta name="generator" content="WordPress 2.0" />
</head>
<body>
<div id="wrap">
<!-- S T A R T T I T L E H E A D B A R -->
<div id="head">
</div>
<!-- E N D T I T L E H E A D B A R -->

<!-- S T A R T L E F T S I D E B A R -->
<div id="left">
</div>
<!-- E N D L E F T S I D E B A R -->

<!-- S T A R T M A I N C O N T E N T -->
<div id="main">
<?php header("HTTP/1.1 404 Not Found"); ?>
<div id="content" class="narrowcolumn" align="center">
<br><br><h1 class="center">I am so sorry. I got caught in the act.
<br>I am not perfect, but I do try.
<br>Please pace your self with a
<br><br><a title="The Good Coffee Place" href="<A href="http://www.coffeecoffee.be/">Good">http://www.coffeecoffee.be/">Good Cup of Coffee</a>
<br><br>to get a fresh start.</h1>
</div>
<p>
</p>
</div>
<!-- E N D M A I N C O N T E N T -->

<!-- S T A R T F O O T E R B A R -->
<div id="bot">
Copyright . All Rights Reserved </div>
<!-- E N D F O O T E R B A R -->
</body>
</html>

[Matts]

I agree with TOPS30, I don't think it's wordpress, and believe it's your webserver setup. As TOPS30 said in the first recommendation, contact the hosting company and tell them to turn off the 404 handler. Do it in the morning with a fresh start.

[miko67]

Ok I investigated a bit further.

My host has a 404.html page for all pages.

I should mention that I am on a subdomain with http://www.coffeecoffee.be/ pointing to it.

On other subdomains I go straight to hosts 404.html page, but on this one with wordpress installed (with permalinks and all), I initially got the wordpress template index page with a post loop telling me "sorry, no posts" if I made a 404_qwerty.

Now I'm thinking to try and make something like this in the top of the template index file:

<?php if (have_posts(0)) { ?>
<?php header("location: /404.php"); ?>
<?php exit(); ?>
<?php else; ?>
<?php } ?>

but I cannot get the code to work so that it goes to my custom 404.php file when there are no posts and still works fine if there are posts.

I strongly believe that I'm a schmuck at php and that it could be rather easy to make the script send the user to my 404.php page if there are no posts, and to run through the normal index header, body and all if there are.

Going to bed now (it's 02:45 am and I have got the kids in the morning).

I am Going to check up on this first thing in the morning though.

[enQuira]

If you are just trying to process the G sitemap verification, just rename the .htaccess, verify then rename it back (or simply turn the rewriting off)

[miko67]

Actually I went over to the php section of dp and got the answer I needed.

See this thread for a working solution: http://forums.digitalpoint.com/showthread.php?t=50963

[baybossplaya]

is there a way to make the 404 redirect to the index?

[miccy]

i think it got something to do with ur htaccess me too having the same problem with my japanese av blog

[seocipl]

Is present an approach in the direction of making the 404 forward toward the index ?

[sellhousefast]

I had this problem too. Thanks for the advice

[sriraj46]

200's means a file found and 400's means a file not found..
And now a file found on a file not found. Interesting. Never came across such errors.

[robinko]

404.php file


<?php get_header(); ?>

your message here

<?php get_sidebar(); ?>
<?php get_footer(); ?>