Possible to induce spiders to crawl?

Hello all,

I have been linkbuilding quite fervently and have recently gotten a few PR6 links which have not yet been crawled/recognized by the search engines. A few (ambiguous) posts on other forums have indicated that it is possible through the use of scripts or SEO tools to induce spiders to crawl pages.

Does anyone have any information on this? Your input is much appreciated as the backlinks I have are from high PR sites but seem to be crawled very infrequently.

Thanks in advance!

 

is the site a .com, .net. org?

 

Possible to induce spiders to crawl?

I find it helps if you pull off several legs.

 

i think there is no tool or script for that

 

You need to promote that link. Ever think about writing a blog post that includes that link? Or add it to your DP sig. Not your site, the link itself.

 

No way to inspire a crawl. Are the pages where the links are cached? If so they know about them. Just because they don't show up in a link: operator or in webmaster's tools doesn't mean anything. That information is deliberately deceptive.

 

Both the site linking to, and the site being linked are .com's.

I'm not worried that they know about them - I'm worried that it could take upwards of a month for spiders to crawl the page (I hear it can take up to that much for a "normal" indexed page which is updated infrequently?).

Thanks for the responses! I'll keep checking to see if there are any more.

 

A PR 6 site is crawled several times a day, another thing is the page your link can be found on, or is it a homepage link?

If you wish the page with your link beeing spidered fast, there are two ways to get the bots to that url, I'll keep the second one for myself, sorry too good and open for abuse.

But a known technique is to write a review on a blog with a link to the page your link is found and ping the blog.

 

Blog & Ping is a technique more used by spammers. It would work good in this case.

I posted a solution for another problem quite similar (a blog not getting too many pages index). This technique can be used for a variation of things.

I use this for the following:
*Increment page counts
*Increment Crawl Rate
*Increment Internal Links

I will re-post it here, so excuse me for duplicating it. Just try it and let me know how it does in a couple of days.

****** Content Map *****

Here is a good technique I use and it help me a lot.

Use Xenu (Google it) It will create a report. From that report highlight all the links that are formatted with title.

Paste this on a html with no format just a title tag. Call it contentmap.html (or any other extension)

Drop it on your hosting, and link it in the bottom of your site (main page/index) with the anchor as content map.

Any one can try this. I have seen my page count increment within days.

Addition: Also I have seen an increment on crawl ratios.

Note: Besides all your sitemaps. No format or anything on this page, just plain links. It won't look good but it doesn't need to look good. Is for the crawlers.

The only downside of this is that you will need to update it manually. Xenu will save you time.

**************

 

I think it will take more time to recognise

 

I had a similar problem and stuck the links back to my site in various places like my dp signature and other places for it to get sucked in..

decodethis above was spot on...