Debt Consolidation - Military Loan - Debt Consolidation - Myspace Layouts - Bet365 bonus
PDA

View Full Version : Slurp doesn't get past homepage?

caroline
Jul 17th 2005, 11:57 am
Yahoo only has my homepage indexed for a drawings site I have! Which ranks well for the terms of the page - which is great and all, but why won't the spider crawl the rest of the site? I guess it could be a simple matter of time, but do you know of any things I could do to speed up the process?
crazyhorse
Jul 17th 2005, 12:08 pm
Just give it a bit more time. Any internal links from the homepage deep linking into the website? Get some links from websites that get spidered more ofthen by Yahoo. Maybe add a RSS yahoo feed? Do you refersh content once a while?

But i guess overall you need to be more patient. In the meantime build on the links, content etc.
Canadianbacon
Jul 17th 2005, 5:25 pm
yahoo is the worst out of the big three for this. It may take months.
caroline
Jul 18th 2005, 5:56 am
Maybe add a RSS yahoo feed?

You mean a RSS feed with recently added/ updated pages? I could do that.
crazyhorse
Jul 18th 2005, 6:17 am
Maybe add a RSS yahoo feed?

You mean a RSS feed with recently added/ updated pages? I could do that.

Yes have a look here at Yahoo RSS (http://my.yahoo.com/s/about/rss/) Constantly changing/update content seems to be to get spidered more often/likely.
I, Brian
Jul 18th 2005, 7:32 am
If you have a Yahoo! account, add the feed to your account - that has been reported as helping speed up the spidering process.

Hang on - will do a test...
SEbasic
Jul 18th 2005, 7:55 am
Hang on - will do a test...It's worked for me :)
wrmineo
Jul 18th 2005, 8:19 am
Yahoo only has my homepage indexed for a drawings site I have! Which ranks well for the terms of the page - which is great and all, but why won't the spider crawl the rest of the site? I guess it could be a simple matter of time, but do you know of any things I could do to speed up the process?

What is the website in question - it may help to look at the site to see if there's an issue going on versus just simply posting guesses as to why it's not being indexed.
webmistress
Aug 27th 2005, 7:46 am
Yup, adding a Y! RSS feed to your pages will speed up spidering at least twofold.
markhutch
Aug 28th 2005, 6:41 pm
Several years ago, Slurp would only request the robots.txt and index page when they had kicked you out of their index. The base of this spider is still the old Slurp, so it could be that if the URL has been around for awhile. Also, if the URL is new, you might want to check the waybackmachine and see if someone else owned that domain name in the past. You could be paying for someone elses sins. It has happened before!
webmistress
Aug 29th 2005, 4:50 pm
Hey Caroline,

You have 3 options:

(1) Wait for Y! site explorer to come live and submit it from this console. http://sitexplorer.search.yahoo.com/

(2) Contact Yahoo search about this problem and in a few weeks they will respond

(3) Pay for an inclusion in the Y! directory. if your site has any penalty they'll remove it or else they won't list you.

I recommend option (1) though