Dmoz Has 10,000+ Dead Links

Surely, Q, you remember, or you might even have bothered to read the thread, that search is not in the live directory and so it will still be showing results.

I have just checked over a dozen of the removed sites and found two new homes, sadly that means 10 unique sites now lost to surfers.

 

They should use some type of script which auto-removes dead links

 

We do, it removes them from the directory you see, but they are still seen by editors so that the editors can ensure the bots are right and also look for an alternative. Didn't I post that earlier in the thread, which means you didn't bother to read the thread either.

Q, your memory really is very bad, perhaps you need to see a doctor, I am worried about you, it wasn't Jim who picked up and ran with those other sites.

Q, you could also have checked to see if the sites were still listed in the directory before you came here with duff information.

 

I have to agree with you, there is about 200 of them inside DMOZ. They are called senior editors.

 

Google did rm -rf for geocities the very next day , forcefully clearing its cache, which otherwise lingers on for months to to come
dmoz should have done the same

 

Hi Jim

What is reason many editors don't delete dead links and they are approving most of the site in few weeks or months?

 

Editors remove dead links from the live directory that you see as they find them. Usually dead links are found by our bots removed from the live directory and then checked to see that they are dead and no alternative sites available by editors.

We can't always detect every dead site and so we are grateful if people using the directory who find dead links report it so that an editor can review it. Sorry don't understand that second half of your question, most people complain that suggested sites are not added quickly enough (though we don't set out to offer the service of listing suggestions, they are just suggestions) We have also been criticised for wanting to spend time sorting out any new alternatives to the geocities sites rather than dealing with suggestions.

edit to add

Some one else not reading the thread, they have been removed from the live directory but are being checked by editors to find alternatives, did Google do that?

 

Their search is usually weeks out of date (like everything else at DMOZ), so it is better to use Google Search Site option.

Now I wonder when will all those possibly unstable deeplinks to wikipedia be removed? As well as numerous other websites which have so many possibly unstable deeplinks listed in DMOZ.

 

Seems like google has already removed them from their directory, that was quick.

 

Doesn't surprise me. Websites come and go every day.

 

Couldnt you just just write a quick PHP (Or any other platform really), to grab all the geocities links, And just check where they resolve to. If you get the "Does not exist" or whatever is coming up for now defunct geocities sites. Then give a SQL call and wallah. All done.

EDIT: I can now see you trolling someone above me for saying the same thing. I still dont get what an editor can do. If they try and view the site and its down, what are they going to do? How can they do anything more then just let the bot delete it?

 

Like all dead or seemingly dead sites the bots and/or editors [depends on who finds it] remove the site from the live directory. Sometimes that turns out to be a mistake or turns out to be a temporary error, sometimes if you wait long enough the site returns. But once a site has been removed from the live directory, it still remains visible to editors, who then check that it is down and then look to see if the site or the information is being held on another site, because an editor has already believed that the site was a valuable asset in DMOZ.

I have spent some time last night working through a few of the geocitioes sites because they have now been removed from the live directory. There is no point in checking them to see if there is a mistake, as we would normally do for sites that have been thrown out by the bots, but I did find two from 12 where the site owner had transferred to a new site and so I was able to put that information back into the directory. I then did not abandon the other sites, but will check in a few months time to see if I can find an alternative. Not every editor will keep them, some editors will search and will delete from the editor directory if no alternative is found.

Google can afford just to remove the sites from their directory because they know very well that if a replacement can be found by a human search then it will appear in DMOZ and when they chose to update Google Directory it will appear there because it is a copy of DMOZ.

Edit to add
If you wonder how we can look for an alternative to a dead site, don't forget we have a title and sometimes the site may have been in a regional locality. So while finding "Gambling for everyone" may be difficult, finding "Naomi Gambling Casino" in London is an easier prospect. But for the first suggestion don't forget that the Wayback website holds a lot of information about a dead web site which can help to trace it.

 

instead of hunting for alternatives won't it be a better idea to go through other submitted sites in same category ?may be they offer what you are looking for


and there is no way to say for sure that "NaomiGamblingCasino.com" is same as geocities.com/NaomiGamblingCasino
unless the old geocities page is itself being redirected to the new domain

 

If the original content about Naomi Gambling Casino is gone and we've found alternative listable content, does it matter who actually owns that content? Our objective is to serve the surfer, not the website owner.

 

And how much "Naomi Gambling Casino" Pays for that service or is it one the senior editors sites?

 

yes, that's why I said

 

You think there are two of them? Incidentally that is a name I invented and not a real site. We can often identify sites by things like contact numbers, addresses and the like (which exist in Wayback) so it is not very difficult to identify if another site is just a replacement.

Usually an editor will look to see if an alternative site has been submitted to the suggestions pool, but hopefully it would have been given as an update which would attract earlier attention. If the pool contains other sites that have similar material that we have not found in the routine checks we do on dead sites, it will be found at some other time when the suggestions are reviewed. Believe that the suggestions pool might have hundreds of sites, so we are now checking 10,000 dead links by looking say in the 100 suggestions and you talk 1,000,000 possibilities, that's even more work. But it sounds to me that it was a webmasters comment that wants the suggestions looked at rather then the way we work which is not to delete an editors work without some checks to see if the site is listed under another URL, or the content of that old site.

 

lol ... so you gonna check these 10K links in waybackmachine for contact and address and than search if those have migrated to some other place

really you guys are real genius!

 

In some of my categories there are sites that I am not happy to lose, so I shall check to see if I can replace them. You spend your spare time doing what you want to do with it, yet you seem to have a problem when I want to spend my time doing what I want. Do I insult you for the way you spend that spare time, like writing cryptic comments on here?

 

OK you can continue to play with your toy i.e. dmoz

OK, I never knew that editors don't have to follow any policy ... they only do what they are happy doing ..no matter how absurd it is