Scrub The Web ending by June 18, 2017

Feel sorry and sad to see getting close one more search engine https://www.scrubtheweb.com

You can view this message on Scrub The Web:
Announcement! After more than 21 years on-line, Scrub The Web will officially shut down on June 18, 2017.
All gone: DMOZ Directory, Yahoo Directory, Alltheweb, Altavista, and now ScrubTheWeb.

Who do you think is to blame social media and phone app where people don't search for the resource anymore or those search engines and directories were not doing right things to keep it up? What do you think?

 

Mostly, it effect to all the business who does not follow the time and changes.

 

Example: Duckduckgo has chrome extension, an app for Android and iPhone, and a team that makes changes as feature comes out. But, DMOZ, Yahoo Directory, Scrub the web never made any app or Chrome extension to make people keep coming back. Dmoz could have made money because people were dying to get a link back from DMOZ and Yahoo Directory neither was ignored by Google nor panelized and they were making around 20 -30 thousand dollar worth listings every day.

2014 Google said link building is not effective, but 2015, 2016, and 2017 Google added few SERP factors but never ignored link building. Google changed the word "link building" to "quality link building".

2018 SERP factors will be out for "voice search engine ranking" so site owners would have to pay attention to that, but the owner does just doing what they were doing in 2001 then they would not get traffic and business. Simple, the business owner needs to change the way they do business and marketing by the time changes and learn what is their competitors are doing to win the market this year or following year.

 

I used both ScrubTheWeb regularly until last year, and I still used DMOZ regularly when it closed. In fact, 1-2 years ago ScrubTheWeb was my 1st choice for searches.

DMOZ had a nice ontology. I could pick an interesting sounding topic, then narrow the results down to find highly relevant sites and articles. Google and other search engines still don't work as well, tho their results may be more up to date.

I liked the fact that ScrubTheWeb didn't use popularity to rank search results. I won't say that listing highly cited sites has no value, but it doesn't seem fair. There are relevant sites which are buried by search engines simply because they're less popular. Listing highly-cited websites separately from regular results would be a better solution. I only stopped using ScrubTheWeb regularly because they started to require JavaScript.

DMOZ's editors are planning to relaunch the directory at Curlie.org; ScrubTheWeb appears to be gone for good.

My Opinion on Why They Closed:

DMOZ was meant as a non-commercial alternative to Yahoo's directory. It was probably the last relic of a world where search ads were BAD, a world where the OpenText project could be drummed out of the search business for adding paid results. In a double twist, DMOZ launched (as GNUHoo) the same year Goto showed everyone that paid results were ok. DMOZ merged with AOL fairly quickly after it launched. At the time it was a good fit. AOL used volunteers to moderate their message boards. These were part of AOL's list of online services; their users were already paying to access them. Using volunteers to organize a list of websites made since. Fast forward 15+ years and AOL's business model is very different. They still have a few subscribers, but their media platforms are dependent on ads. A free directory which had "we'll always be free" written into it's bylaws was no longer a good fit. Besides, Google's updates over the last few years have devalued directory links. Google even said a DMOZ link wasn't that important. Turning it into a pay-for-entry directory service wouldn't have worked. And Verizon is in the middle of a disastrous-sounding merger with Yahoo. Did I mention that Verizon bought AOL a year or so before they decided to buy Yahoo? I'm pretty sure their investors are pushing them to cut costs.

ScrubTheWeb made money from their SEO services, including a directory/search engine submission service. Google's recent updates (again) have devalued such services. Over the past few years they've had fewer visitors; they probably lost a lot of customers too. No Customers = No Money = Closed Business.

P.S. DMOZ did have a toolbar, for Firefox and IE, if memory serves, and ScrubTheWeb launched a new version of their site in October of 2016.

 

Hi @pacman000, Nice information. Are you an editor or employee at DMOZ or Scrub the web?

 

I was not an employee of ScrubTheWeb, but I did email them about a year before they closed to find out more about their history. How they made money and why they closed is speculative; tho there was a message to their registered users stating the engine cost too much to run.

I volunteered to edit a small video game related category a month or so before DMOZ closed. Got approved, cleaned it up, got a second category, then a 3rd before it closed. No one seems to know why AOL decided to drop the directory; this was my speculation. The bit about the directory always being free was part of the directory's social contract, which was available to the public; the bit about OpenText, Goto, and GnuHoo as a Yahoo alternative is just search engine history.

 

It is up to them, but we are adding 4 more language and better design in 3-4 weeks for Bhanvad.com

 

Who's next, Such a shame...

J

 

Can you explain, what do you mean by "such a shame?"

 

Yes, (thank you sarahk) That's what i meant, Sorry cm2010, Not aimed at Bhanvad.com by any means.

Best Regards
j

 

Thanks to Sarahk and Phplduser. It created misunderstanding because reply came after we announced of creating in more languages.

 

No worries cm2010, Have a good day!

J

 

I wrote an article on ScrubTheWeb's history, what little I could find of it: https://websitering.neocities.org/ScrubTheWebIsGone.htm

 

most likely they did not keep up with the trends ie: new technology...
nothing on scrub ever seemed to change, everything old, smells old....
even if they were updating internally to some degree, it takes time, and
money to keep up, and I would imagine it is hard to commit those resources
if you see your business going in decline. I think it would be safe to say if
they were making money, then they would have stayed open or even sold...

I certainly would not mind having that high authority domain that
they probably could get some good money for i bet, even with their
high authority domain, it was not enough to save them....

like the posts above, times change, people stop using the old, and
start using more trendy services....

It reminds me of those SEO guys that still use the same SEO crap they
used 5 years ago. If you cannot adjust, and keep up with google
guidelines, then sure enough google will slowly run you out of town.
You cant keep running and selling those same crash and burn services
that were once popular 5 years ago....

then again, directories are not the only thing that has taken a heavy
hit either, forums are not like they used to be because of FB; forums
used to be very popular, not so much anymore, damn FB groups....

 

@dscurlock

Ah yes, their advice seemed old. Even five years ago it seemed old. That's why I was surprised that their search engine worked fairly well. Keep in mind that results vary across queries; few people are trying to spam "Atari" or "Comics." Even when I found a set of results which were a bit too commercial or embarrassing it wasn't too hard to create a more specific query to get better results. "Video Game Reviews" instead of "Video Games;" "Comics -adult." (Probably 1 of 10 or 1 of 20 results for art-related queries were from adult sites; not as bad as EuroSeek in 1999, judging from what's in the Internet Archive, and ScrubTheWeb did claim to be censor free.) They had good support for Boolean phrases, better than what Google offers today, which tends to ignore part of my query and return less precise results.

Their search engine submission service wasn't advertised as a way to get hundreds of backlinks, but I'd guess sure that's what most people used it for, even if many of these services would only return a link once a user entered a query. Most of their other tools had free equivalents elsewhere, or came in a package with other services, like rank trackers, which ScrubTheWeb didn't have. (A rank-tracking program would violate other search engine's TOS; that's why ScrubTheWeb didn't have one, according to their FAQ.)

They did put their domain for sale; it was linked from their home page for awhile. They said they wouldn't sell their software or customer list. Selling the customer list would violate their privacy policy; I don't know why they didn't want to sell their software. As a business decision it makes little sense to sell the domain name if they had plans to continue as something else, licensing the software for enterprise search or something, but as a personal decision I guess I could understand it.

Thanks for responding.

 

There are several different factors that have contributed to the situation where online directories are dying. The example of printed yellow pages can be brought into this discussion. When people nowadays look for a car mechanic, they do a search on the Internet. Before Internet, there was a market for the yellow pages, because they offered a convenient way to get the information you wanted. But now even online directories are not doing well. This can mostly be attributed to how well search engines do their indexing today. Look at Google Business. Search for a car mechanic followed by your city and you will find the information you want. There's no need for you to consult an online directory.

People want things to be convenient, and this is offered by search engines. Just an example of another thing that is taken over by search engines. Before when you wanted to read a songs lyrics you would need to visit a lyrics website. Nowadays Google will display the whole lyrics. So there's no need to visit the lyrics websites. The question is following. When the search engines upgrade themselves to the better, can the online directories keep up with the tempo?

 

I noticed Bhanvad is listed as a partner on ScrubTheWeb's new directory. Did you buy the domain, cm2010? Do you know who did/if it really sold?