I came across an article titled Del.icio.us Leaves a Bad Taste (Aug 30th), which suggests that the Search Engines do actually spider links on del.icio.us. This is contrary to the observations of the robots.txt file and meta robots tags which are clearly excluding all bots from all content. The theory goes that these robot flags are dynamically changed based on the user agent, and certain search engines are permitted to crawl the site after all. This can be easily seen by entering the search query "site:del.icio.us". Google shows 3 million results for that domain, while Yahoo shows 1.5 mil and MSN 250 thousand. However, I couldn't find any specific person's bookmarks page (i.e. mine) indexed on any search engine. Perhaps a del.icio.us page needs to have a link to it (or submitted) before it will be spidered... Someone has suggested that Yahoo spiders links from del.icio.us more regurlary and faster than it's own submission service (source). There also the problem that links on del.icio.us are tagged as no-follow. Perhaps these get removed for certain search engine bots too? Interesting, huh. Cryo.
I've had my links indexed by Y! from the tag (.com/tag/keyword) page on many sites. Also, there are a lot more social sites to go after. Del.icio.us gets updated so fast, sometimes the SE's don't ever get to them before their buried. Check out www.TagAutomator.com for a usual tool to make updating to these social sites a little easier.
After looking into this some further, it seems that del.icio.us tag all outbound links with rel="nofollow", even if the user agent is GoogleBot. This means that those links are not counted as backlinks. Quote from Wikipedia: Spam in blogs: This suggests that while links from del.icio.us are not counted as backlinks, they will be spidered by the bots. So, if you have a new site that you want to get indexed quickly, make sure that a) your del.icio.us profile is linked to and spidered, and b) your new site appears on it. Cryo.
Further update: From my own experiments, Yahoo now show links from my del.icio.us boomarks as backlinks to my domain. So, there is a benefit after all! There is also Yahoo's own version called "My Web" (they bought del.icio.us, remember), where you can add links and share them with other people. Here's mine, for example. I've yet to see if there's any thing to gain from this - some people reckon that Yahoo will index sites faster if listed on My Web... The good news is that there is not a single ref="nofollow" in sight, which is one difference. Meanwhile, Google has indexed (and cached) my del.icio.us page, and all of the sub pages (tags, categories, etc). This only happened when I linked to it from another page. Too early to say if these will count as backlinks, but I doubt they ever will. MSN have only indexed my top level del.icio.us page, but does not have it cached yet. Cryo.