Is there a secret to getting Yahoo to "come back" and crawl my site again? Here's the situation: when I launched my site 3 months ago, I had many pages in the root folder. Shortly after Yahoo crawled my site and indexed everything, I restructured my folder hierarchy and moved all those pages into a subfolder, but I did it (what I thought was) "the right way" - I copied the pages to their new location, updated my internal links and sitemaps, set up 301 redirects to point the old addresses to the new addresses, and disallowed access to the old addresses in robots.txt. This should have theoretically eliminated all access/visibility to these pages from good robots like Yahoo's. Google completely cleaned up its index by removing the old and indexing the new, but Yahoo is still showing the old pages without removing them from their index. Is there any way to get them to wake up and smell the coffee? There are too many pages (about 100) to use the "Remove URL" feature in their Site Explorer (which doesn't work anyway due to a CSS problem that prevents you from clicking the Yes button, but that's another topic for another day). If I delete the sitemap from Site Explorer, wait a few days and add it back, will that do the trick?
It's been 2½ months since I made the changes. However, I think I (might have) figured it out. Turns out the redirects I was using were 302 redirects and not 301's. I changed them to 301's, so now I will wait some more and see if that has any effect. Maybe they can send me some traffic too. Haven't had a single visitor referred from them, but hundreds from Google.
Make sure everything you do is legal and the content is yours and not copy pasted from other site. Fresh content and quality backlinks are the only secret. Also Social Media is one of the determining factors as well.