SE's getting 302 but should get 404

[kaptain]

I deleted a (huge) directory about 7 months ago at that time i also created a custom 404 page. The SE's still have all these pages indexed :rolleyes:

I noticed yesterday that my 404 had <meta name="robots" content="index,follow"> in the heading tags.

So i changed it to "noindex".
Would that be a explanation why SE's still have the deleted directory indexed?

Also when the SE's are crawling this site when looking for that directory they get 302 errors before they get a 404. Why?

[tomecki]

I think, that if you delete your custom 404 page you should have it shortly unindexed (firstly in google).

[J.D.]

Would that be a explanation why SE's still have the deleted directory indexed?

Also when the SE's are crawling this site when looking for that directory they get 302 errors before they get a 404. Why?

Many SE's will visit 404'ed URLs for as long as there are links to these pages. I have seen pages gone for about a year and still crawled by Google. It's a good idea to configure your server to return 410 (Gone) for pages that are gone forever.

As for 302, it seems that your server is simply misconfigured and somehow treats your 404 page as a regular file-based page and reports to the browser that it didn't change. You have to make sure that the HTTP status code returned with this page is indeed 404 (i.e. that it's not just HTML saying that the page is not found).

J.D.

[kaptain]

i must have misunderstood something about creating my own 404.

What i did was create a page directly from my site template and added some text to explain the page had changed. Also had the standard heading tags.

Then added the 404 redirect in the .htaccess

So it seems as if the SE's thought all the pages i deleted got redirected to the custom 404.

So... i did delete the custom 404 and now it returns the standard 404. I checked my log file and now the SE's are getting the 404 error for that directory.

Thanks,

One more question, when you have say... 100,000 pages return 404's (the directory i deleted) could that affect my SE rankings?