Google Deep Crawl Frequency

[krilin50]

Hi to everyone!

I am a bit puzzled regarding G Deep Crawl. I have spent hours on the internet searching for a way to forecast when G will crawl my site, so that I will have time to make the necessary adjustment and updates to the site. Nobody has been able to provide me with an answer. The only info I could find regarding this was that it would depend on PR and some other details. Nevertheless, I met this "SEO expert" who was able to tell me when the next crawl was going to happen and to my amazement he was correct!! was it pure luck? or this guy really knows what he is talking about. I would appreciate if anybody could shed some light into this.

Many thanks

[leo]

I made the following observation: As soon as I have done several (even minuscule) adjustments to my page(s) over about a week's time, G increases its crawl frequency to - at the moment - daily visits to all pages. If I stop that, crawls will diminish to, first, daily visits of e.g. index and Sitemap only and then, upon further reduction of my own activity over two or three weeks, the crawls are further reduced to every two or three days and so on. Over the past half year I had at least two visits per week even during my most inactive periods (holidays).

In conclusion, G seems to adapt it's crawl frequency to what seems actually needed due to actuality of the information offered. What counts is simply date of upload fo the corresponding pages - I do not have the impression that content change really matters a great deal.

[leo]

This seems to be a double thread!

[disgust]

it depends. there is a monthy deep crawl for ALL sites, but small sites see crawls all the time depending on their PR.

the last one was around the start of this month I believe (maybe even a week before?). it should be between 20 and 30 days from then, usually

[Chiara]

I don't know why or when it happens, but Google deep-crawled me today. I think I need a cigarette.

[T0PS3O]

Double indeed: http://forums.digitalpoint.com/showthread.php?t=1122

[GuyFromChicago]

Originally Posted by Chiara

I don't know why or when it happens, but Google deep-crawled me today. I think I need a cigarette.

I think two things impact the frequency of the "deep crawl":

Links pointing to your site & fresh content.

[Foxy]

I launched a site about two weeks ago

It was crawled in hours and for a certain keyword phrase was no1 before 12 hours was up.

How did this happen?

Well first it was linked from a "authority site" that was being crawled frequently because of the factors listed above in the other posts - and because it was new it was fresh - but it only got one crawl.

Next day it was back and did a little more and then back again on the same day ie twice in one day - at the moment it is crawling daily - but the content is changing daily also.

[digitalpoint]

Originally Posted by leo

This seems to be a double thread!

Yep, and I merged them into a single thread.

[leeds1]

Foxy is right

a) Get your link from a site that has a regular crawl (normally PR5+)
b) Change your content regularly

Foxy - how do you find authority sites ?

I do get crawled, probably 2/3 times per day at the mo - and this has been going on for weeks. I do have a forum so that helps in terms of fresh content.

Just looked at my logs and G has made 11138 hits on my site already this month and taken up 248.05 MB of bandwidth

[Foxy]

Foxy - how do you find authority sites ?

Look for size, Site PR, frequency of Googlebot visits [which you must judge] and overall theme.

For example this forum is more authoritative than seo.... than liquid markets than abacus, at this moment in time

How do I know this? because if you do a check on "weber bbqs" you will find this forum 2+3 seo.... 13+14 liquid markets 17+18 abakus 26