When google employees started posting years ago that submitting to directories was a valid way to get your site indexed and ranked on google, could they have had a more sinister intent? Let's paint another picture... What if Google had not said anything about directories, and more and more directories like Yahoo, DMOZ and business.com continued to grow their traffic? Then fewer people might use a search engines, and they might be inclined to use more directory type sites like Yahoo. Keep in mind many of these directory type sites also went on to develop their own content too, and Yahoo was a dominant player. By insuring that any large scale attempts to make quality directories failed, Google could increase people's dependency on their search engine, and keep people away from human edited sites. By allowing some directories to game pagerank, google could rotate through which directories were on top and insure there was never a dominant player. The directories that gamed pagerank would peel off some of the business from quality directories, thereby destroying a large part of the revenue stream of Yahoo, business.com and others. Putting DMOZ right into google was a brilliant way to further weaken Yahoo. I say I am at least mostly right. What do you say?
Let me add they seem to be doing the same thing to Wikipedia right now. Google says they like unique content, but are thought a destroyer of quality sites if they aren't owned by Google?
And then of course there is what Google did to Yelp and Trip Advisor http://bits.blogs.nytimes.com/2012/08/16/a-window-into-what-its-like-to-compete-with-google/ They first encouraged the sites only to get the spammed to oblivion, and then start their own site. It seems Google "says" they are looking for great content, but then if anyone gets good at that, they orchestrate a loss of revenue for those sites. Exceptions might be site's like Amazon that pay Google big money for advertising, but you never know if Google might be planning to take Amazon down one day too. And speaking of New York Times articles, did Google put up news.google.com to minimize the importance of newspapers and then go on to penalize some of them for selling links?
It's hard to fathom the idea that anyone would WANT to use a web directory over a search engine to find information they're looking for.
What would you classify yelp as? (there isn't a right or wrong answer just interested to see your perspective). I quite often these days find myself avoiding Google completely in favour of particular sites, depending on the topic... An example? There's a site called Basketball Reference. A friend was trying to work out which NBA player had played for 4 different teams in his career, under 4 different numbers. It was for a competition to win something. He immediately guessed Dennis Rodman (who if I'm not mistaken played for 5 teams with 4 numbers). Using Google proved difficult. However Basketball Reference had each players jersey and the number they played in, below their profile, so you could see at a glance who and what and when and where ---- If you wanna know the answer I will tell ya It's interesting to me that a lot of times I find myself ON a directory after using Google's results first (having gotten there via the SERPs).
The idea which you have pointed out seems quite interesting. Although I doubt if it is true! Search engine is more powerful than directories serving user's quest.
It's a site that lists bars, restaurants, in a city via guess what... categories! The key ingredient to the site is human reviews. i.e. People giving their real stories about their experience at said bar/restaurant with a rating. To my way of thinking it is a directory, it just doesn't fit in that narrow-minded idea of what people think a directory is (static categorised links only). That's an old idea, the web has evolved, and so have many directories.
For sure they still serve a purpose and can be of value when done right. I've always thought there was a future for niche directories but let's not kid ourselves the majority of them are spam/low quality that only exist to sell PR or transfer link juice to the owner's websites. But going back to the topic of this thread, I think it's pretty ridiculous to think Google had sinister intentions and had it in for directories. Search was always going to be the dominant player.
The problem with the directories that you mention and the other larger ones is that they often degrade to lists of dead links. More intent on adding new links than auditing their existing ones, some of the sites keep on growing but become less useful with time. DMOZ lost a huge amount of its credibility when so many of the editors were exposed for demanding payment for inclusion in their assigned areas. Combined with the huge amount of links still pointing at Geocities and other disbanded, yet easily cullible, sites; it's become less of a player with each passing year. I just went on business.com to try and find the content writing category. Never did find it. Found a technical writer via a search with results that said they were listed under Web Design Services - didn't find them there either. It's also not where I would look for a technical writer. On the web design services page there are 49 listings - most would agree that's too many outgoing links on one page and that doesn't include what look like affiliate links at the top. 17 of the 49 have "web design" in the title - very few have that phrase within their company name or URL. The phrase is used 44 times within the listings. And I didn't count all the times "website design" or "web site design" was used. Additionally, a lot of the listings throughout the site have anchor like "somecityname Web Design Company", "Web Design anothercityname", etc. Could it be that the over-occurrence of a few targeted keywords and a bit of over-optimization is at fault here? Or maybe too many outgoing links on the same page? (each listing has two for the same URL => 98 outgoing links) I know with a niche directory I once consulted on, keyword overuse was an issue - almost every listing had the phrase "transportation company" in them - cleaned that up and the penalty went away. Sure, it's hard to not reuse the same phrases in a category where everyone offers basically the same thing - but, with some writing skill, it can be avoided. The other option would be to modify the way the content is provided and perhaps use some graphics to indicate what services are provided. For a site that charges $299 a year, you would think they could figure a way around that plus provide/require better descriptions than... DMOZ, Business and some of the other biggie boys are definitely huge. But, I think they have become too much for the editors to handle effectively. For years, it didn't matter as much if a site had too many links using the same phrase. Now it does. Having huge advertising budgets to build a brand and buy a fancy dictionary URL are not going to count as much as the content does anymore. Yahoo, in my mind, is a special case. It has been a competitor of Google's from day 1. I have no doubt Google hopes it will go away sometime soon just like Walmart hopes to see the demise of Kmart and Target. Yahoo has made a number of mistakes and has not always met the needs of users as the Internet changes and grows. They seem more concerned with news and social stuff than they do with indexing the web. It shouldn't come as a surprise to anyone that their $299 directory listings and search results have declined in popularity. How many times have you actually used their directory? Whereas I have used DMOZ from time to time, I think I can count how many times I've used Yahoo or Business on one hand. Wikipedia is a curious example and one I'm not entirely sure Google knows what to do with. In an update where top quality content-based sites got a boost and WP started dominating the SERPs, webmasters complained and complained loudly. Another update later and some of the WP results slid back down. As much as I hate to admit it, why should my article on a famous painter with a handful of links outrank one on WP with thousands of backlinks and a full bibliography? On the other hand, if Google didn't let some of the smaller players into the mix, they might as well just index WP and forget listing anything else. This one doesn't have an easy answer. I've noticed the new data being served by Google. I like it from a researching point of view and it does seem to add value. Sure, it will decrease visits to the sites listed. If a search for when Michelangelo was born includes a sidebar with the information I need, why do I need to visit any of the listings? But, I've been doing that sort of search for years and just taking the information from the descriptions and not visiting the sites; so I'm not sure the net result is really all that different. I could easily justify the move as simply providing information. There's no doubt that it's bound to decrease traffic to everyone. An assault on Wiki and the rest of us, I tend to not think so. However, if I want to learn about who the artist was and what he painted, I'm more likely to just start my search on Wikipedia in the first place. While I have no doubt that WP gets many billions of hits from Google searches, in my mind it is a bit Google-proof in that even if it were ever delisted, people would still use it. I know I would. Google seems to allow sites at all ends of the spectrum defy their policies. I've seem competitors with obvious networks, improperly placed Adsense and pathetically weak content come up high in the results all the time. And, of course we all know of one very outspoken directory owner who proudly admits he's gaming PR. Some eventually fall, others never do. Whether it's by design or that they just haven't figured out an computer-driven, non-human edited way to spot them is anyone's guess. In many ways, I see Google as the web equivalent of WalMart. As they continue to look for ways to expand, they are going to buy out or destroy others. We either have to continue changing to offer something unique or get left behind.
Let's keep in mind that google has a history of using humans to improve their search too. This is not a practice limited to directories. Google just want to make sure that other human created solutions never get large enough to compete with their ad delivery engine which also includes a few organic results as well.
Google has human reviewers to improve the quality of it's SERPs. The same can't be said about directories that will gladly accept your link for payment. I know you're the owner of the phpLD script, and seriously most of what is coming out of your mouth is just a bunch of sour grapes. You've been in this business long enough to know you have to adapt to the changes. The last year especially has been crazy in the SEO world.
Wikipedia brings a new problem, in that, the content can be changed by anyone. I've been to movie listings (e.g.) that went months without being altered, when someone changed a correct listing to be false (and even derogatory vandalism). This happened on a page with citations, so it's like Wikipedia is saying "this is fact, and here is the evidence why"... but the listing was incorrect, and was left that way for months. I went back in and changed it/corrected it, to what it should be. But for several months that information was incorrect. So really, how is Wikipedia any more useful than anything else? It isn't it can get it just as wrong, and does so from an authoritative status. Don't get me started on the "this article needs a cleanup" stuff. Those ones should be avoided for all purposes because you don't know what's real and what is made up. They do use humans... Just probably not in the way we think they should (or do). Once a company goes from being a cute little startup to a corporation with a board of directors and shareholders to please, you can't really blame them for losing the "do no evil" mantra and adopting a "win at all costs" one instead. They are in this business to make money. If they let every little upstart take a slice of the web, that is future profit they are giving up. It makes sense that they compete the way they do. If they don't they'll be yesterday's news. I can't fault them for what they do from their perspective, any corporation would do the same. It is why people think corporations are evil. It is all about the profit and who cares who gets destroyed on the way to achieving that profit. Your example isn't really indicative of the industry in which we speak. Those sorts of directories (the entire solicitations and announcements forum even) can be dismissed with a wave of the hand. That's not the sort of business we want or need. But by the same token, you are making out like Google is doing what they do to improve the SERPs. Why are they no better (and in many cases far worse) than pre-Panda and pre-Penguin? Think back to when Google said you couldn't sell pagerank. I have no problems with that, but you know what they turned around and did? They allowed businesses to sell it, if they did so via adsense. Meaning, look you can sell PR-links (yup paid links that send juice) transparently, if you use our Google Adsense system and we get a cut of the action. That's a different perspective to what they were saying don't you think? They tell webmasters/SEOs things from this ethical standpoint, but that standpoint is false, what they are really saying is "we have to penalise you because you are encroaching on our business, which in turn affects our profits, and therefore upsets our shareholders... We need to penalise you because if we don't we lose out... Cough up some cash and we'll turn a blind eye though!!!" How else can they justify the "sponsored listings" in the SERPs. Those positions technically aren't "ranked" higher but they sit higher than the organic listings... it's pretty underhanded don't you think? Or do you think that's legit, from the ethical standpoint?
While some people have said there are ethical challenges to this, I have been a proponent of better educating the public about Google being Pay Per Click. There is nowhere on the search results pages telling people the arrangement google has with the advertisers, specifically that the advertisers pay per click. While click fraud is illegal, I do consider it within a consumers right to "charge" the advertiser for putting the ad there via a click. It would be an interesting dilemma if people were told that the opponents to their political candidate in the elections could be effectively charged for posting advertising in support of their candidate if they clicked on the opponent's paid ads. If people started to be more activist about clicking on ads, it would dilute the effectiveness of ads on google, and make organic searches more important. In essence, the more we click on things we would not necessarily buy, the less effective PPC becomes, and I don't think there is anything wrong about making that information public. In my personal opinion that is not click fraud to make people aware that more frequent clicking can lead to lost effectiveness of ads they don't like. Where it would become click fraud is when someone purposely tries to take down their competitor. But it becomes much more difficult to establish a case if someone for example was frequently checking a site for updates, or clicking more ads than the average consumer. There are definite threats to the google business model, and educating the public about the ads is very ethical in my opinion. People should be able to make decisions based on facts, and there are some missing facts about how the advertising works on the google results pages.
This all depends on how a business sees it's PPC campaign going. If it sees a lot of clicks it probably thinks it is successful (even though it doesn't realise that all of those clicks were done by people not even slightly interested in their website and who basically click-and-closed). Unless they are actually going into in-depth analysis about their visitors (and I bet most companies don't even bother), they wouldn't realise that their campaign is actually counter-productive because their traffic is increasing even if their sales are not.