DMOZ and rel="nofollow"

...and, to add to what Bradley said, that utterly defeats the main objective of the ODP - to provide a list of human-edited URLs to anyone that wants it.

 

Okay, point taken, although I'm not sure I'd characterize it as a "cheap dig" rather than an observation.

Phrase it this way instead:

If that were done, it would not only reduce "spam" - it would also substantially reduce any interest outside of the DMOZ group itself in anything to do with DMOZ.

 

Holy crap! Look at all the DMOZ editors waiting in line in this thread!

 

Just to clarify this (since there are now two separate issues being discussed in this thread), rel=nofollow does not disable a "live" link - it just instructs spiders not to follow that link.

 

And where does that understanding come from? I would propose that most editors have no clue what PR is, let alone what the PR of the links from the listed sites are.

 

Minstrel, I was following up on Bradley's post, not the general topic of the "nofollow".

I maintain that adding a nofollow tag to the ODP website would do nothing to affect Google, and therefore make no difference to editor's motivations for being a part of the ODP. And I have yet to see any argument that counters that.

 

Why would we want to reduce interest in DMOZ? Those of us working there believe it's a good thing and we want it to be seen.

Yes, and we still haven't heard how Will comes to his conclusion that corrupt editors will quit. Still waiting for that explanation before I even consider accepting that "nofollow" is a good idea, even though I don't exactly understand the technology of it. Macdesign's explanation makes me think it can't be done. Is he wrong? If so, then why?

 

Thank you for the clarification, I did't know that.

This is true. Discussions revolving around PR are way over my head. Sorry, but you're dealing with a lot of editors who don't care about PR...maybe we would if we understood it, but I doubt it.

 

More than one editor has said they see DMOZ as designed for web surfers, not webmasters. I doubt that there is any real interest in DMOZ beyond webmasters and DMOZ editors but if you really want to discourage webmasters' interest, eliminating most of the PR factor would definitely do that.

If there were no PR value from a DMOZ listing, corrupt editors would have no motivation to remain: (1) listing their own sites and trying to prevent listing of competitors' sites would have no value, and (2) trying to "sell" a DMOZ listing wouldn't work because the market would vanish.

I doubt that but regardless - most of the interest in DMOZ (beyond editors) comes from webmasters looking for the PR boost and that would disappear or be greatly reduced.

I wasn't suggesting that you were confused. I was clarifying it for other readers.

 

OK, fair enough. What I don't see is how adding "nofollow" tags to the ODP web pages would do anything to affect Google's PR. That is the leap of logic that I don't understand.

The discussion about whether it is something that the ODP would WANT to do is a different issue entirely. Right now I can't even see it providing the functionality that Will seems to think it would, for the reasons which have already been given by others. At the very least it would be temporary (while Google is sipdering ODP pages directly) and would disappear as soon as Google does its Directory update (which it presumably spiders, too)

As for sites gaining credibility by having a listing in the ODP, I don't have a problem with that at all. I think it's a valid conclusion in most cases. A human has looked at the site and (assuming they are not corrupt) has made a decision that at the time of review it conformed to the ODP guidelines for unique valid content in a subject.

The idea that the ODP should reduce usability of the directory in order to try to reduce editor corruption and submission spam is a case of cutting off your nose to spite your face, I believe.

I have concluded from previous posts that nddb, Will and others wish to try to reduce the effect that the ODP has on the SEO world. I believe I understand why they are trying to do that.

I believe that Google uses the ODP (in whatever form it does) as a trusted source for human-reviewed sites. They do not believe that it is fundamentally corrupt, horrendously out-dated, or yesterday's news or any of the other things I have seen levelled at the ODP in the many threads that exist (if they did, they wouldn't use it). Instead it is one good source of data upon which they feel they can rely. Asking the ODP to try to cripple that would be counter-productive for Google and the ODP.

 

Please excuse my confusion, but after letting this sink in my head is about to explode with stupid questions.

I thought:
1. Spiders are used by search engines, not directories.
2. ODP is a directory, so we don't use spiders.
3. Google and others take ODP data from some sort of "dump" that doesn't include html.

Are any of these three items true, or am I mistaken?

Now macdesign and bradley have mentioned "scrape code" and "scraper site technology" which I haven't heard of before today. How does this affect spiders, directories, and nofollow tags?

In my current frame of confusion, I still can't see where nofollow tags would help even if they are possible, and I don't know if they are possible.

I understand what alucard is saying, and I understand part of what the others are saying, but minstrel, you evil genuis, you've given me a headache!

 

Search engines use spiders to find pages on the net. What spiders do as a matter of course is follow links from one page they know about to other pages they may not know about, so that they can add those pages to the SE index. It doesn't natter where they find those links, on a single website or in a directory, it has the same effect - they will try to follow those links out to the other pages unless instructed not to do so. That is their default behavior.

Probably true (for all I know) but we aren't talking about DMOZ spiders... we are talking about SE spiders, especially Googlebot.

Since I've never tried to take a DMOZ dump, I don't know... perhaps this is true.

 

Oh I am SOOO not going there!

 

At least give me credit for being able to post that with a straight face



(and thanks, Annie, for being the Martin to my Rowan, or the Laurel to my Hardy, or the Trigger to my Roy Rogers, or... well, never mind )

 

Ok, getting better...

So, google takes ODP data from the dump, and google also sends a spider to dmoz.org? Isn't that a bit redundant? It's the same information.

This means Google takes the data from the dump (which cannot have nofollow tags) and makes its own directory (without nofollow tags) AND sends its spider to the dmoz.org site --so even if there were nofollow tags in the actual dmoz listings, the listings would be free of the tags in googles own directory.

I still don't see how adding nofollow tags in the ODP directory would make a bit of difference since the ODP directory is only one small part of where the listings are harvested.

According to the link furnished by macdesign, nofollow in ODP, the dump is XML and cannot pass along the nofollow tag. (btw, good article, thanks for the link.)

Do I understand this correctly? Or is it beyond my understanding?

 

No. You're on the right track, Annie.

Will's suggestion is only part of the answer. It would send a message I guess from DMOZ about their purpose - and their attitude about the use of the directory by webmasters. It would seem to remove one source of criticism of DMOZ, perhaps. And it would remove one source of PR.

As long as Google takes DMOZ dumps and values them, and as long as they give value to links on all the DMOZ clones out there (which may no longer be happening actually) there is still a problem. Will's suggestion would be a step in the right direction but would not entirely resolve the problem,

 

I am still not seeing this.

If the "problem" is that Google derives some of it's PR from ODP data, then I don't agree that that's a problem, for reasons I have already given and which I know aren't shared within this community.

But even if I did, Google using the RDF as a basis of finding sites would bypass any penality that the nofollow on the dmoz.org pages has...

 

But does Google do that? I don't know...

 

So let's see if we can summarize this:

1. We don't know what Google do to arrive at PR for a site
2. The RDF wouldn't be influenced in any way by the "nofollow" tag
3. IF the Google Spider uses the ODP website as a basis to find and rank sites, then possibly the nofollow tag would negatively impact the PR rating of listed sites for a short while.
4. If Google perceives the ODP as a trusted source of data, it might then build an exception into its Google Spider to ignore the "nofollow" tags in the ODP web pages.
5. And why should this be done? To go a very small amount towards fixing a perceived problem of corruption in the ODP, which has not been proven to be as rampant as most ODP-detractors would like the world to believe.

I'm sorry, but so far I am not seeing a compelling argument for the ODP to cripple its website....

If you believe that the ODP is not a good source of information about content on the internet, then put your compelling arguments to the ones that use it - Google. I don't think you are going to get very far, though.

The ODP believes that, while not being perfect, it is the best out there. Editors are proud of that (otherwise they wouldn't be editors). Editors will strive to improve it even further. And that includes sniffing out corrupt and self-interested editors who damage the quality of the data in the directory.